Kotaemon

Last reviewed Aug 26, 2026 · 791 words

Kotaemon's advantage over every other self-hosted "chat with your PDFs" tool is what happens after the answer: click a citation and the source page opens beside the chat with the supporting passage highlighted. That is the feature that makes retrieval trustworthy for real work, and it is why the project reached 25,715 stars within a couple of years of its 2024 debut. The price is a heavy Python stack, Medium difficulty, and a 4 GB minimum that covers the application alone, before any model runs. Treat it as the front end of a small RAG stack, not as a single app.

Lite or full is the first decision

Cinnamon publishes two container tags: main-lite and main-full. Lite handles text PDFs, Office files, and plain text with the standard loaders. Full adds Unstructured and OCR dependencies for scanned documents, images inside PDFs, and messy layouts, and it is several gigabytes larger. If your documents are born-digital, lite is enough and far quicker to start:

docker run -d --name kotaemon \
  -e GRADIO_SERVER_NAME=0.0.0.0 -e GRADIO_SERVER_PORT=7860 \
  -v ./ktem_app_data:/app/ktem_app_data \
  -p 7860:7860 ghcr.io/cinnamon/kotaemon:main-lite

The interface is Gradio on port 7860. The ktem_app_data volume holds the vector store (Chroma by default), the document store (LanceDB by default), uploaded files, and the user database, so it is the only thing to back up. Multi-user mode with an admin account is switched on through the environment, and the default admin credentials must be changed before anyone else can reach the port.

Models: local through Ollama, or an API key

Kotaemon needs two models, an embedder and a chat model, and it is agnostic about where they come from. The simplest fully local setup points both at Ollama using its OpenAI-compatible endpoint, with nomic-embed-text for embeddings and a 7B to 14B instruct model for answers. That brings the real memory requirement to the 4 GB for Kotaemon plus whatever the models need, so 16 GB is the honest floor for a smooth local experience and a GPU makes the difference between answers in seconds and answers in a minute. The hardware for self-hosted LLMs notes size that properly. Pointing the chat model at a hosted API while keeping embeddings and documents local is a legitimate middle path if the documents are not sensitive but you do not own a GPU.

Pipelines are configurable, and mostly should be left alone

Under the hood is a reasoning pipeline you can swap: simple retrieval, a ReAct agent that calls tools, and GraphRAG variants (the project bundles LightRAG and nano-GraphRAG integrations) that build an entity graph from your documents and answer relationship questions a flat vector search misses. The graph modes are impressive on a corpus of related reports and expensive to index; a few hundred pages can take an hour and burn a lot of tokens if the model is remote. Start with the default pipeline, hybrid search on, reranking on, and turn on graph indexing for one collection where the questions are genuinely about how things connect. The RAG versus fine-tuning piece explains why retrieval is the right tool here in the first place.

Against the neighbours

AnythingLLM is easier to install, lighter, and has workspaces with a friendlier admin experience, but its citations point at chunks, not highlighted pages. Open WebUI does document chat as one feature among many and is the right choice if you mostly want a general chat interface with occasional file upload. Kotaemon is for the person whose work is the documents: a researcher, a lawyer, someone with a shelf of technical manuals, anyone replacing NotebookLM who wants the sources on their own disk. The NotebookLM alternatives page sets that comparison out.

Operational notes

The Gradio front end has no built-in TLS; put it behind a reverse proxy and, since Gradio's own auth is thin, forward-auth from your identity provider. Pin an image tag once you have a working setup, because the dependency stack is large and upgrades have moved defaults before. Re-indexing after an embedding model change means re-uploading; there is no automatic migration of vectors.

What I'd do

main-lite, Ollama on the same box with nomic-embed-text and a 14B model on a 12 GB GPU, one collection per project, default pipeline with reranking on, graph indexing for a single corpus where it earns its cost, reverse proxy with forward-auth in front, and nightly backup of ktem_app_data. If the box has no GPU and 8 GB of RAM, run AnythingLLM instead and come back when the hardware catches up.