Qdrant

Last reviewed Aug 26, 2026 · 867 words

One million 768-dimension vectors need about 3 GB of RAM in Qdrant before the index, and the planning number is 1.5 times that. For the 20,000-chunk personal knowledge base most self-hosters actually build, the same sum gives 60 MB, and the catalogue's 512 MB floor is comfortable. Do that arithmetic before anything else, because it decides whether Qdrant lives happily next to Ollama on the same box or needs a machine of its own, and it explains every "why is it slow" thread on the forum.

Size the box: vectors, dimensions, 4 bytes, plus half

A float32 vector costs 4 bytes per dimension. Multiply by the number of points, add roughly 50% for the HNSW graph, and that is the working set. Three knobs change it. on_disk: true on the vector config keeps vectors memory-mapped and pages them in, trading latency for RAM. Scalar quantisation to int8 cuts vector memory by 4x with a small recall hit, and binary quantisation cuts it by 32x for models that tolerate it. Payload (your metadata and text) is stored on disk by default and does not count. So a 5 million vector collection that would need 20 GB raw fits an 8 GB box with quantisation and memory-mapped originals, at a few extra milliseconds per query.

Compose, API key, dashboard on 6333

services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "127.0.0.1:6333:6333"
      - "127.0.0.1:6334:6334"
    environment:
      - QDRANT__SERVICE__API_KEY=replace-with-40-random-characters
    volumes:
      - ./qdrant_storage:/qdrant/storage
    restart: unless-stopped

6333 is REST and the built-in web dashboard at /dashboard; 6334 is gRPC, which the official clients prefer for bulk upserts. Bind both to localhost or a Docker network and let your reverse proxy add TLS if anything outside the box needs them. Without the API key variable, anyone who can reach the port owns the data, and the "easy to deploy" pro is only true once that line is in place. A collection is one HTTP call:

curl -X PUT http://localhost:6333/collections/notes \
  -H "api-key: replace-with-40-random-characters" \
  -H "Content-Type: application/json" \
  -d '{"vectors": {"size": 768, "distance": "Cosine"}}'

The size must match your embedding model exactly: 768 for nomic-embed-text, 1024 for bge-m3, 384 for the small MiniLM models. Get it wrong and every insert fails with a clear error, which is the best kind of wrong.

Payload filtering is the reason to choose it over pgvector

If you already run Postgres, pgvector is a fair question, and for under 100,000 vectors with simple queries it is a fine answer. Qdrant pulls ahead when filters are part of the search: "nearest chunks where source = 'wiki' and year >= 2024" runs against payload indexes during the HNSW traversal rather than as a post-filter that silently returns fewer than k results. Named vectors let one point carry a dense and a sparse (BM25-style) embedding for hybrid search, and the Query API fuses them server-side. Multitenancy is a payload field plus a filter, which is how a household or team shares one instance. The vector database comparison goes through Chroma, Milvus and Weaviate on the same axes; the short version is that Qdrant is the one that is both simple to run and hard to outgrow.

Where it sits in a local RAG stack

Ollama produces the embeddings (nomic-embed-text runs on CPU at a few hundred chunks a minute), Qdrant stores and searches them, and a chat front end does retrieval and generation. Open WebUI uses Chroma by default and switches with VECTOR_DB=qdrant plus QDRANT_URI and QDRANT_API_KEY; AnythingLLM and Dify have Qdrant in their vector store dropdowns. For your own code, the Python and TypeScript clients are thin wrappers over the REST API, and the whole loop (embed, upsert, query with filter) is about 40 lines. Latency on a home box is single-digit milliseconds per query at the sizes discussed here; the embedding call dominates, not the search.

Backups are snapshots, not a copy of the volume

Copying qdrant_storage while the process is running produces a backup that may not load. The supported path is the snapshot API: POST /collections/notes/snapshots writes a self-contained file you can download from the same endpoint, and a fresh instance restores it with PUT /collections/notes/snapshots/upload or the --snapshot flag at startup. A full-storage snapshot exists too. Cron the collection snapshot nightly, ship the file with restic, and test the restore on a scratch container once; the cost is a few minutes and the snapshot for a 20,000-chunk collection is under 100 MB.

What I'd do

Qdrant in compose next to Ollama with the API key set and ports on localhost, one collection per embedding model, on_disk: true only if the RAM sum says so, and a nightly snapshot to the backup repository. Skip Pinecone unless you need a managed SLA. If your data is small and already in Postgres, pgvector is acceptable; the moment filters matter or the count passes a few hundred thousand, move.