Choosing a Vector Database: pgvector Until It Hurts

Use pgvector until one of four numbers turns red: more than ~20M vectors in a table, sustained search traffic in the hundreds of QPS, filtered queries where recall visibly drops, or HNSW rebuild windows you can no longer schedule. Most RAG applications never hit any of them — a million-vector HNSW index in Postgres answers in single-digit milliseconds — and the cost of a dedicated vector database is not the licence, it's operating a second stateful system forever: backups, upgrades, monitoring, and a distributed consistency story your team now owns.

What pgvector actually handles

The capacity conversation goes better with concrete numbers. One million 768-dimension float32 vectors is about 3GB of table data plus roughly 4GB of HNSW index; halfvec (fp16, added in 0.7.0) halves both with recall loss that's typically under a point. Query latency at that scale is 1–10ms p50 with the index in RAM, which is the operative condition — HNSW that pages from disk falls off a cliff, so RAM sizing is the real capacity plan.

CREATE INDEX ON chunks USING hnsw (embedding halfvec_cosine_ops)
  WITH (m = 16, ef_construction = 64);

SET maintenance_work_mem = '8GB';  -- before the build, not after the OOM
SET hnsw.ef_search = 40;           -- recall/latency dial, per session

Two version-specific facts that decide borderline cases. First, index builds are the hidden cost: an HNSW build over 10M vectors takes tens of minutes and wants maintenance_work_mem sized to hold the graph — undersize it and the build spills and slows dramatically. Second, pgvector 0.8 added iterative index scans, which fixed the worst filtered-query failure (previously, a filter that excluded most of the HNSW candidates could return too few rows; now the scan keeps pulling until it satisfies the query). Filters that keep maybe 1% or more of rows now behave reasonably; needle-in-haystack filters still favour a dedicated engine. The pgvector README is genuinely the best single document on all of this.

The strongest argument for staying is the one nobody benchmarks: your vectors live in the same transaction as your data. Delete a user, and their chunks are gone — no dual-write pipeline, no sync job that drifted, no GDPR-deletion audit spanning two systems. If you're already running Postgres for everything, that's worth real QPS.

The four break points, specifically

Scale. Past ~20M vectors, keeping HNSW in RAM alongside your OLTP working set turns into a memory bill (20M × 768d in halfvec is ~30GB of index) and rebuild times move from coffee to change-request territory.

Throughput. Vector search is CPU-hungry, and in Postgres it's stealing cycles from your transactional load with no isolation between them. A few hundred sustained QPS of ANN search on the same box as your app's database is where p99s on both start telling on you. Read replicas buy a multiple, not an order of magnitude.

Filtered recall. If every query carries WHERE tenant_id = ? AND doc_type = ? and selectivity is high, you want an engine whose index was built for that.

Operations. If reindexing after an embedding-model change (which rewrites every vector — see embeddings search in production) can't fit your maintenance windows in Postgres, a store with non-blocking index management earns its keep.

What Qdrant buys you when you move

Qdrant is my default second step because its strengths map exactly onto pgvector's break points. Filterable HNSW is native: payload indexes participate in graph traversal, so high-selectivity filters don't crater recall or latency. Built-in quantisation attacks the RAM problem — scalar int8 cuts vector memory 4× for a recall cost usually under 1% (and you can rescore with originals); binary quantisation goes up to 32× and works surprisingly well on high-dimensional embeddings with oversampling. Snapshots, collection aliases (blue-green reindexing while serving), and horizontal sharding cover the operational gaps. The Qdrant documentation is unusually honest about which knob costs what.

What you give up: transactional coupling with your relational data, SQL joins, and one fewer 3am-pageable system becomes one more. Budget the dual-write pipeline and its failure modes into the comparison, because that's where the real cost lives.

The decision ladder

Benchmarks that matter, benchmarks that lie

Vendor QPS charts are the ones that lie: measured on datasets that aren't yours (ann-benchmarks corpora are mostly ≤1M vectors and low-dimensional by 2026 standards), at recall settings buried in a footnote, with no concurrent writes. QPS without a recall number attached is marketing.

The four measurements worth an afternoon, on your own data: recall@10 against exact brute-force search on a 1,000-query sample (build the ground truth once, it's cheap at evaluation scale); that same recall with your real filters applied at your real selectivity; p99 latency under concurrent ingest, because compaction and graph maintenance are where engines differ most; and steady-state RAM at your target recall. Any engine can win a benchmark where recall is allowed to float — pin recall at 0.95 and compare what each system costs to deliver it.

What I'd do

Default: pgvector with halfvec and HNSW on the database you already run, ef_search tuned against a measured recall target, and a calendar reminder to re-check the four numbers quarterly. Move to Qdrant when filters or RAM force it, migrate one collection at a time behind an alias, and keep the relational source of truth in Postgres with vector IDs pointing at it. And whichever engine you pick, spend the afternoon on your own recall benchmark before believing anyone's chart, including mine.