Infinity: a single-binary search engine for vectors, tensors and text

What problem Infinity is aimed at

A retrieval-augmented generation pipeline usually needs more than one index. Dense embeddings handle semantic similarity, BM25 handles exact terms and rare tokens, and a multi-vector representation such as ColBERT keeps token-level detail that a single pooled vector throws away. Running those as three services means three schemas, three write paths and a merge step in application code. Infinity's pitch is that all of them live in one table: the README describes support for dense vector, sparse vector, tensor, full-text and structured data, and states that hybrid search across those types is supported in addition to filtering. The audience is engineers building search, question answering or recommender features who are willing to run their own search process rather than call a hosted API. The README frames the target as LLM applications and RAG specifically, and the repository topics include rag, bm25, hnsw and hybrid-search, which is consistent with that framing.

The mechanism: one table, several index types, a reranking stage

The data model is a table with typed columns. In the README's example, create_table takes a mapping of column names to type descriptors: {"num": {"type": "integer"}, "body": {"type": "varchar"}, "vec": {"type": "vector, 4, float"}}. The vector type string carries dimensionality and element type, so the index is built per column rather than per collection. Queries are built by chaining: output(["*"]).match_dense("vec", [3.0, 2.8, 2.7, 3.1], "float", "ip", 2).to_pl(). The arguments name the column, the query vector, the element type, the distance metric (ip, meaning inner product) and the result count. Results come back as a Polars dataframe through to_pl(), which is a deliberate choice: the client hands you a dataframe rather than a bespoke result object. Hybrid search is the part that differs from a plain vector store. The README lists rerankers including RRF (reciprocal rank fusion), weighted sum and ColBERT, so the intended flow is to score candidates from more than one matcher and then combine those scores in a final stage inside the engine. What the README does not specify is how the fusion stage is configured at the API level, which arguments select RRF versus weighted sum, or how per-matcher weights are expressed. That detail sits in the linked Python and HTTP API references, not in the README, so treat the README as a map of capabilities rather than a configuration guide.

Getting a server and a client running

The README's primary path is Docker with client and server as separate processes. It first creates a data directory and hands ownership to the current user, then pulls and runs the image with a raised file descriptor limit and host networking: sudo mkdir -p /var/infinity && sudo chown -R $USER /var/infinity, then docker pull infiniflow/infinity:nightly, then docker run -d --name infinity -v /var/infinity/:/var/infinity --ulimit nofile=500000:500000 --network=host infiniflow/infinity:nightly. The nofile bump to 500000 is not incidental for a search server holding many open index segments, and --network=host means the client reaches the server on the host network rather than a mapped port. On Windows the README requires WSL or WSL2, with systemd enabled inside WSL2, docker-ce installed per Docker's Ubuntu instructions, and, for Docker Desktop 4.29 or newer, the Enable host networking option switched on before the same run command is used. The client is installed separately: pip install infinity-sdk==0.7.3. Note the version split. The client is pinned to 0.7.3, which matches the v0.7.3 release listed in the repository, while the server image tag in the README is nightly. Prerequisites are stated plainly: x86_64 with AVX2 support, Linux with glibc 2.17 or newer, Windows 10 or newer through WSL, macOS, and Python 3.11 or newer. The README also points to a binary deployment guide and a build-from-source guide for people who do not want Docker, and notes that Infinity can be embedded in Python as a module.

The AVX2 floor and the single-binary trade-off

The hard constraint is the CPU. Infinity requires x86_64 with AVX2 support. That is a reasonable assumption on server hardware from the last decade, but it is a real boundary: ARM hosts, older Xeons without AVX2, and some virtualized or emulated environments will not run it. The README does not describe a fallback path for those machines, so this is a gate rather than a tuning parameter. The second trade-off is architectural. A single-binary design with no external dependencies is genuinely easier to deploy than a system that expects a coordination service and object storage, and the README calls this out as a deployment advantage. The cost is that scaling out is not described in the README at all. There is no discussion of replication, sharding, or how multiple server processes would coordinate over the same data directory. Mounting /var/infinity from more than one container is not something the documentation shown here addresses. If your requirement is horizontal scale with a documented replication story, this README gives you nothing to evaluate, and you would need the references section to answer it.

Where a plain vector database is the better choice

If your retrieval is a single dense embedding column and a metadata filter, Infinity's breadth is overhead you will pay for without using. A system such as Qdrant or Milvus covers that case with a narrower surface: you define a collection, insert points with payloads, and query by vector with a filter expression. The difference in approach is what the engine is organized around. Those systems center on the vector index and treat payload as attached metadata; Infinity centers on a typed table where a varchar column is a first-class indexed field alongside the vector columns, which is why BM25 and hybrid fusion sit inside the query rather than in a separate search service. That matters when your queries genuinely mix lexical and semantic matching, and it matters much less when they do not. The other axis is operations. A hosted or managed vector service removes the server process, the data directory, the file descriptor limit and the AVX2 requirement from your plate. Infinity's README presents a self-hosted binary and nothing about a managed offering, so choosing it means owning that process.

Performance claims and what to check before trusting them

The README states 0.1 millisecond query latency and 15K+ QPS on million-scale vector datasets, and 1 millisecond latency and 12K+ QPS for full-text search on 33M documents. It also links a benchmark report on infiniflow.org. Those are the project's own numbers under its own conditions, and the README does not state the hardware, the dataset, the recall target, or the concurrency level behind them in the text shown here. Latency figures for approximate nearest neighbor search are only meaningful alongside a recall number, because a lower recall target is cheaper to hit. Before you plan capacity around 0.1 milliseconds, read the linked benchmark page for the configuration and then reproduce it on your own data and hardware. The same caution applies to the hybrid path: the README lists RRF, weighted sum and ColBERT rerankers but does not publish latency for a fused query, which is the shape most RAG pipelines actually issue.

Versioning, licence and upgrade cost

The release list shows v0.7.2 and v0.7.3 as dated releases plus a nightly tag, and the README's Docker instructions pull nightly while the client is pinned to 0.7.3. That mismatch is the main upgrade cost to plan for. A nightly server image moves under you, and the Python client is version-locked, so a deployment that follows the README literally is running a moving server against a fixed client. Pinning the server to a released tag and tracking the client version alongside it is the more predictable arrangement, and the README's own install line for the client shows the project does tag releases. The project has not been archived, and the last push date on the repository is recent, so the codebase is active. On licensing, Infinity is Apache-2.0. That is a permissive licence that generally allows commercial use and modification, but it also carries notice and attribution obligations, and it is not legal advice. If you embed the binary in a product you ship, have counsel review the notice requirements rather than assuming the licence grants more than it does.

Editorial conclusion

Adopt Infinity if your retrieval layer needs dense vectors, sparse vectors, tensors and BM25 text scoring in one query, and you can run a server process on x86_64 with AVX2. Do not adopt it if you only store dense embeddings, or if you need a managed service with per-tenant isolation, because the README documents a single self-hosted binary and nothing else. Verify first that your CPU reports AVX2, that the pinned client version matches the server version you pull, and that the nightly image tag is acceptable for your deployment, since the README's Docker examples pull infiniflow/infinity:nightly while the client is pinned to 0.7.3.

Official sources

1. infiniflow/infinity on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases