Milvus 3.0: A Vector Database That Separates Compute from Storage for Scale

The Problem Milvus Solves

Milvus addresses a specific pain point: storing and searching vector embeddings at a scale that a single machine cannot handle. Applications that generate embeddings from text, images, or multi-modal data need a database that can ingest billions of vectors, keep them fresh with streaming updates, and answer thousands of queries per second. Milvus is built for that scale, not for toy demos. It is aimed at teams running production AI workloads who need more than an in-memory index. The README positions it as a high-performance vector database with a fully-distributed, K8s-native architecture. That positioning is the core of its identity. If you are evaluating it, you are likely dealing with a workload that has outgrown a simple library like FAISS or a single-node store.

Architecture: Compute and Storage Separation

The key architectural decision in Milvus is the separation of compute and storage. The README states that the distributed architecture separates compute and storage, allowing independent scaling of query nodes and data nodes. Query nodes handle read-heavy workloads, while data nodes handle write-heavy workloads. This is a stateless microservices design on Kubernetes, which enables quick recovery from failures. The system also supports replicas, which load data segments on multiple query nodes to improve fault tolerance and throughput. This separation is not just a scalability feature; it changes how you plan capacity. You can add query nodes without touching storage, and vice versa. The trade-off is operational complexity. Running a distributed system with multiple node types and coordinators is more involved than running a single database process. The README mentions coordinator HA, which implies that coordinators are a separate component that can fail and recover, adding another layer to manage.

Getting Started: From Lite to Distributed

Milvus offers multiple deployment paths, which is a strength. For a quick start, you can install the Python SDK with `pip install -U pymilvus`. Even lighter, you can install `pymilvus[milvus-lite]` and create a local vector database by instantiating a client with a local file name: `client = MilvusClient("milvus_demo.db")`. That gives you a single-file database for development. For production, you connect to a deployed Milvus server by specifying a URI and token: `client = MilvusClient(uri="<endpoint>", token="<credentials>")`. The README also references a standalone Docker installation, which sits between Lite and the full distributed mode. The API surface is consistent across these modes, so code written against Lite can be pointed at a server later. That is a practical migration path, but be aware that the deployment mode changes performance characteristics. Lite is not a substitute for the distributed version under load.

Index Types and Hardware Acceleration

Milvus supports a range of index types: HNSW, IVF, FLAT (brute-force), SCANN, and DiskANN, along with quantization-based variations like IVFPQ and mmap support. This breadth lets you match the index to your data size and latency requirements. FLAT is exact but slow at scale; HNSW is a common approximate choice; DiskANN is for data that does not fit in memory. The README also mentions GPU indexing, specifically NVIDIA's CAGRA from the RAPIDS cuVS library. That is a notable feature for workloads that can afford GPU resources. However, the documentation does not specify which indexes support GPU acceleration or how much speedup you get. You would need to check the index documentation for details. This is a case where the README gives a promise but not the specifics. For an engineer, the choice of index is not a minor detail; it affects recall, latency, and memory footprint. Milvus gives you options, but you must understand the trade-offs yourself.

Multi-tenancy and Storage Tiers

Milvus supports multi-tenancy through isolation at the database, collection, partition, or partition key level. The README claims this allows a single cluster to handle from hundreds to millions of tenants. That range is wide, and the actual limit depends on how you configure isolation. Partition key isolation is likely the most scalable, but it also affects search performance because the system must route queries based on the key. The README also mentions hot/cold storage: frequently accessed data can live in memory or on SSDs, while colder data sits on cheaper storage. This is a cost optimization, but it adds a tiering decision. You need to decide what is hot and what is cold, and that is not automatic. The documentation does not describe how tiering is configured or whether it is manual or automatic. This is an area where you should verify the current behavior before trusting the cost savings.

Licensing and Governance

Milvus is distributed under the Apache 2.0 license, which is permissive for commercial use. The project is under the LF AI & Data Foundation, with Zilliz as the major contributor. That governance structure matters if you are concerned about long-term stewardship. The license allows you to modify and redistribute the code, but you should read the license text for specific obligations. The README also points to Zilliz Cloud as a managed service, which is a separate offering. That means there is a commercial path if you want to avoid self-hosting. The open-source project and the managed service are distinct, but the cloud option is integrated into the documentation. If you are evaluating Milvus for a company, the Apache license is a low-risk choice, but the governance under a foundation does not guarantee that Zilliz will not shift focus. The release history shows active maintenance, with v3.0.0 released in July 2026 and v2.6.23 in August 2026, indicating a dual-track release cycle.

Limitations and When It Is the Wrong Tool

Milvus is not the right tool for every vector search problem. If your dataset fits in memory on a single machine and you do not need real-time updates, a simpler in-process library like FAISS or a single-node database like SQLite with a vector extension may be enough. Milvus Lite exists for quickstarts, but it is not intended for production scale. The distributed mode requires Kubernetes and a set of microservices, which is a significant operational burden. The README does not mention any failure modes, but the architecture implies that coordinators are a single point of failure unless you configure HA. The documentation mentions coordinator HA, but that is an optional setup. Another limitation is that the README does not specify the maximum vector dimension or the exact performance numbers. You cannot assume that Milvus will handle your workload without testing. The system also supports metadata filtering and range search, but those features can impact performance if not used carefully. If you do not need horizontal scaling or streaming ingestion, Milvus is overkill.

Alternatives and the Difference in Approach

The most direct alternative is Qdrant, a vector database written in Rust that also supports filtering and horizontal scaling. The difference is in architecture: Qdrant uses a single binary that can run in distributed mode, but it does not separate compute and storage to the same degree. Milvus separates query nodes and data nodes, which gives finer-grained scaling but adds complexity. Another alternative is Weaviate, which is also cloud-native but has a different storage engine and uses a graph-based approach for some features. The practical difference is that Milvus is more explicit about compute-storage separation, which matters if your workload is write-heavy or read-heavy in a lopsided way. Qdrant and Weaviate are often easier to operate because they have fewer moving parts. If you need GPU acceleration, Milvus has a specific feature (CAGRA) that those alternatives may not offer. The choice depends on whether you value scalability over simplicity.

Editorial conclusion

Adopt Milvus if you need a horizontally scalable vector database with real-time streaming updates and a mature ecosystem around Kubernetes. Avoid it if your workload is a small prototype where Milvus Lite or a simpler tool suffices. Before committing, verify your exact index type, GPU support, and multi-tenancy isolation level against the current documentation, and confirm that your deployment environment can handle the operational overhead of the distributed mode.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes