LightRAG

Last reviewed Aug 26, 2026 · 932 words

The bill for LightRAG arrives at indexing time, not query time. Plain vector RAG embeds each chunk once, cheaply. LightRAG additionally sends every chunk through an LLM with a prompt asking it to list the entities and relationships inside, merges those into a graph, then embeds the graph nodes as well. On a 200-page corpus that is hundreds of long LLM calls before you have asked a single question. Against a paid API it is dollars per corpus; against a local 8B model it is hours, plus a quality ceiling set by how well that model extracts entities. The payoff, when the documents have structure worth capturing (people, projects, components, and the links between them), is answers to "how does X relate to Y" that flat chunk retrieval fumbles. HKU's data-science group released it in 2024 under MIT, it has 39,181 GitHub stars, and it runs as a Docker container on 2 GB of RAM because the heavy lifting happens in whatever model server you point it at.

Five query modes, and only two matter at first

Retrieval mode is the main knob you get:

Start with hybrid, keep naive around to check the graph is earning its cost, and switch to mix when answers are conceptually right but cite the wrong number. If naive matches hybrid on your real questions, your corpus does not have graph-shaped structure and you can stop paying for extraction.

The server speaks Ollama, so your chat UI already supports it

lightrag-server exposes a web UI (document upload, graph viewer, query console) and two APIs: its own REST endpoints and an Ollama-compatible chat endpoint. The second is the practical trick. Point Open WebUI at LightRAG as if it were an Ollama host and it shows up as a model; people chat with the corpus without learning a new interface. Configuration lives in a .env beside the compose file:

LLM_BINDING=ollama
LLM_BINDING_HOST=http://ollama:11434
LLM_MODEL=qwen3:14b
EMBEDDING_BINDING=ollama
EMBEDDING_BINDING_HOST=http://ollama:11434
EMBEDDING_MODEL=bge-m3
EMBEDDING_DIM=1024
WORKING_DIR=/app/data/rag_storage
INPUT_DIR=/app/data/inputs

The server listens on 9621 by default. LLM_BINDING also accepts openai, azure_openai, and lollms, and anything OpenAI-compatible (vLLM, LiteLLM) goes through the openai binding with a custom host.

The Ollama context window is the setting that silently ruins it

Ollama's default context for most models is a few thousand tokens. LightRAG's extraction prompt plus a chunk plus its examples is longer than that, and the project's own docs ask for at least 32k. With too small a window Ollama truncates from the front, the model never sees the instructions, and you get an index full of empty or garbage entities with no error anywhere. Fix it before you index a single file: build a model variant with PARAMETER num_ctx 32768 in a Modelfile, then verify with one small document in the graph viewer. Also set EMBEDDING_DIM to match your embedding model and never change the embedding model afterwards without wiping WORKING_DIR; the vectors do not survive a swap. Ollama in production covers the context and concurrency knobs in more depth.

JSON files until they hurt, then PostgreSQL

Out of the box, storage is files in WORKING_DIR: a NetworkX graph and a small JSON-backed vector store. That is right for one corpus and one user, and it survives restarts as long as the volume is mounted. Past a few thousand documents, or with several people querying, swap backends. LightRAG supports PostgreSQL for all four stores at once (key-value, vectors via pgvector, graph via Apache AGE, document status), and separately Neo4j for the graph with Milvus or Qdrant for vectors. The single-Postgres route is the least to operate, and the Postgres for everything argument applies. Whichever you pick, back up WORKING_DIR or the database, not only the input files: re-indexing is the expensive part.

Young and fast-moving: pin the tag

The con in the catalogue is accurate. Releases land often, the config surface has been renamed more than once, and a :latest pull can change environment variable names under you. Pin a specific tag, read the changelog before moving, and keep the compose file in git. Against the alternatives: Microsoft's GraphRAG does a more elaborate community-summarisation pass and costs more to index; RAGFlow is a full document platform with better PDF parsing and an 8 GB footprint. LightRAG is the lightweight one, which is both its name and its scope: bring your own parsing for messy PDFs, and expect to write a little glue.

What I'd do

Run lightrag-server in Docker with a pinned tag beside an Ollama container, a 14B-class model with num_ctx at 32k for extraction, bge-m3 for embeddings, files-on-a-volume storage to start. Index one representative document, open the graph viewer, and only feed it the rest once the entities look sane. Query in hybrid, keep naive for comparison, and move to PostgreSQL the day two people want it at once. If the corpus is mostly scanned PDFs, parse them with Marker or Docling first; LightRAG is the retrieval layer, not the reader.