GraphRAG

Last reviewed Aug 26, 2026 · 778 words

Indexing 100 documents with GraphRAG means several thousand LLM calls, because it extracts entities and relationships from every text chunk and then summarises every community it finds in the resulting graph. Against a hosted frontier model that is tens of dollars per run; against a local 7B model on a single consumer GPU it is a job measured in hours, with noticeably worse extraction. This is the fact that decides whether GraphRAG belongs in your stack, and it is the one the project's README plays down.

What it does that plain RAG cannot

Ordinary retrieval-augmented generation embeds chunks, finds the nearest ones to a question, and hands them to a model. It answers "what does the lease say about termination" well and "what themes run across all 400 support tickets" badly, because no single chunk contains the answer. GraphRAG builds a knowledge graph of entities and their relationships during indexing, clusters it into communities, and writes a summary for each community at several levels. Two query modes then use that structure: local search, which pulls the entities and neighbours relevant to a specific question, and global search, which map-reduces across community summaries to answer the sweeping questions. If your questions are the sweeping kind, the graph pays for itself. If they are lookups, it is an expensive way to get a slightly worse answer, and the RAG vs fine-tuning piece covers cheaper routes.

The install is a Python package, not a server

GraphRAG is a library and command-line tool, so self-hosting it means running it on a machine with Python 3.10 or newer, about 4 GB of RAM for the pipeline itself, and access to two model endpoints: a chat model and an embedding model.

python -m venv .venv && source .venv/bin/activate
pip install graphrag
mkdir -p ./ragtest/input   # drop .txt files here
graphrag init --root ./ragtest

init writes settings.yaml and a .env holding GRAPHRAG_API_KEY. The defaults target OpenAI. To point at a local Ollama instance, change the model blocks to an OpenAI-compatible base URL, for example api_base: http://localhost:11434/v1, set the chat model to something like qwen2.5:14b and the embedding model to nomic-embed-text, and lower the concurrency settings so a single GPU is not hit with 25 parallel requests. Then:

graphrag index --root ./ragtest
graphrag query --root ./ragtest --method global --query "What are the main themes?"
graphrag query --root ./ragtest --method local --query "Who works with Alice?"

Output lands in ./ragtest/output as parquet files: entities, relationships, communities, reports. A vector store is part of the pipeline; the default is a local LanceDB directory, and you can swap in Qdrant or another store if the corpus outgrows one machine.

Local models work, with lowered expectations

The extraction step asks the model to return structured entity and relationship tuples from each chunk. Frontier models do this reliably; 7B and 8B local models drop entities, invent relationships, and occasionally break the output format, which the pipeline retries. A 14B or larger model in the Qwen or Llama families is where results become usable, and a 32B model is where they become good. That is a 24 GB GPU for any reasonable speed. Budget the indexing run as an overnight job and treat re-indexing as rare: incremental updates are supported, but a changed prompt or model means a full rebuild.

Where it does not fit, and what to run instead

The tooling is research-grade. There is no web UI, no user management, no scheduler, and settings change between releases. If you want "chat with my documents" for a household, AnythingLLM with plain RAG delivers 80 percent of the value at 5 percent of the compute. If you specifically want graph-based retrieval on a homelab budget, LightRAG and nano-graphrag reimplement the core idea with far cheaper indexing and are friendlier to local models; LightRAG in particular ships a working server mode.

What I'd do

Try GraphRAG once, on a corpus under 50 documents, against a hosted model with a spending cap of 20 dollars, to see whether global search answers questions plain RAG could not. If it does and the corpus is stable, run the real index as a one-off on rented GPU time and serve the results locally; the query side is cheap. If the corpus changes weekly, or the questions turn out to be lookups, use LightRAG on Ollama and keep the money. GraphRAG is the reference implementation in the AI category, not the practical one, and it is best used as exactly that.