Qdrant, Chroma, Milvus or pgvector for a first RAG prototype

Open SourceEric3 repliesThis topic is about qdrant/qdrant

EricMember#1

The vector database choice gets more attention than it deserves for a prototype, and less than it deserves once the prototype survives. The four that come up most, as tracked here:

Chroma (Apache-2.0). Python-first, runs embedded in your process with a PersistentClient pointing at a directory, no server to run. The fastest way from zero to "it retrieves". The server mode exists for when you outgrow that.

Qdrant (Apache-2.0, Rust). A single binary or container with an HTTP and gRPC API. Its strengths are payload filtering (filter on metadata inside the vector search rather than after it) and quantisation options that cut memory. The point at which Chroma users tend to move.

Milvus (Apache-2.0). Built for distributed deployments: separate components for coordination, storage and query, backed by object storage. Heavier than a prototype needs; Milvus Lite (pip install) exists for local use with the same API, which is a sensible on-ramp if you expect to need the full thing.

pgvector. Not a database but a Postgres extension. If your application already has Postgres, this is the answer for the prototype and quite far beyond: HNSW indexes, and your vectors live next to the rows they describe, with joins and transactions. The ceiling is lower than a dedicated engine, and most projects never reach it.

Our pairs: /compare/chroma-vs-qdrant, /compare/milvus-vs-qdrant. Background: /learn/vector-database, /learn/embeddings. To get an intuition for the distance metric, /tools/vector-similarity-calculator lets you paste two vectors and compare cosine against dot product and Euclidean.

3 replies

Devn NieMember#2

Strong second for pgvector when Postgres is already there. The underrated advantage is operational: one backup, one set of credentials, one thing that pages you at night. A separate vector store is a separate thing to run, and for a prototype that is the cost that kills it, not query latency.

DaisyRegular#3

Where do keyword search and hybrid retrieval fit? Pure vector search misses exact identifiers (error codes, part numbers, function names), which is most of what a developer-facing RAG gets asked. Does any of the four do BM25 alongside vectors, or is that a separate component?

EricAuthorMember#4

In reply to Daisy (#3)

Qdrant and Milvus both support sparse vectors, which is how you get BM25-style matching inside the same engine and fuse it with dense results. Chroma has added full-text filtering. With pgvector you have Postgres's own full-text search next to it, and you combine the two rankings in SQL. So it is available everywhere now; the difference is how much of the fusion you write yourself. For developer documentation your instinct is right: hybrid is not optional.