Letta

Last reviewed Aug 26, 2026 · 838 words

Letta's whole idea fits in a sentence: the agent's context window is not a transcript, it is a small working memory the agent rewrites itself with tool calls, backed by a larger store it can search. Everything else, the server, the Python and TypeScript SDKs, the visual editor, is scaffolding around that. It started life as MemGPT, a 2023 research project that framed the context window as RAM and the database as disk, and the rename to Letta came with a proper server you can run at home. It is Apache-2.0, Python, 24,442 stars, and the catalogue calls it Medium with 2 GB of RAM, which is honest as long as the model runs somewhere else.

Three memories, and the agent manages two of them

A Letta agent carries "core memory" inside its prompt: a handful of labelled blocks, by default one describing the agent's persona and one describing the human it is talking to. The model has tools to append to or replace text in those blocks, so when you mention that you moved to Lisbon, a well-behaved agent edits the human block rather than hoping it stays in the chat history. "Archival memory" is a vector store the agent writes to and searches when a question needs older facts. "Recall memory" is the full message history, also searchable. The trick is that the model decides what to promote and what to forget, which is what makes an agent feel like it knows you after a month and what makes a weak model produce nonsense: it has to reason about its own memory, not just answer.

The server is one container plus Postgres

The reference deployment is a single letta/letta image with a bundled Postgres and pgvector, exposing port 8283:

docker run -d --name letta \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  letta/letta:latest

Swap the environment variable for OPENAI_API_KEY, ANTHROPIC_API_KEY, or a vLLM endpoint and the same server drives agents on whatever model you point it at, per agent, so one agent can plan on a hosted frontier model while another runs entirely on your GPU. The persisted volume holds every agent, its memory blocks, and its history; back it up like any Postgres directory. The visual Agent Development Environment is a web app that connects to your local server over that port, so you get the point-and-click editor for memory blocks and tools without sending your agent state anywhere except the model provider you chose.

Small local models are where it hurts

Memory editing is tool calling with judgement, and 7B-class models make bad librarians. In my experience an agent on a small quantised model will forget to save the fact, save it into the wrong block, or call the search tool for every message. The catalogue lists "best with capable models" as a con and that is the whole story: a 30B-class instruction model on a 24 GB card is where a fully local Letta agent stops being a demo. Ollama works as the backend, and the hardware for self-hosted LLMs post covers what those model sizes cost to run. If your only GPU is an 8 GB card, run the Letta server locally and point it at a hosted model; the memory still lives with you.

Where it sits in a stack, and what it is not

Letta is the agent runtime, not the chat UI and not the retrieval layer. The usual homelab shape is a Letta server, a model backend, and something that talks to Letta's REST API: a Slack or Matrix bot, a small web page, or an n8n workflow that hands it messages. It is not a document RAG system; archival memory is a vector store, but you would still want a proper ingestion pipeline for a thousand PDFs. It is also not a multi-agent orchestrator in the way the agent architectures post uses the term, though agents can message each other. If all you want is "remember facts about the user" bolted onto an existing chat app, Mem0 is the lighter library; Letta is for when the agent itself is the product.

What I'd do

Run the container on the same box as Ollama, build one agent in the visual editor with a persona block you actually wrote, and let it run on a hosted model for a week to see what a working memory agent feels like. Then try the same agent on the largest local model your card holds and see whether the memory edits still happen. If they do, you have a private assistant that improves with use; if they do not, keep the server local and the model remote, which is still a better privacy position than any hosted assistant product. Either way, back up ~/.letta before you get attached to it.