infino

What infino does

infino is a fast retrieval engine that runs SQL, full-text search, and vector search over a single copy of your data on object storage. The data stays in Parquet on S3, Azure, GCS, or local disk, and you query it at scale. The project optimizes for speed per dollar, making tradeoffs to achieve object-storage economics at search engine speeds. The README notes that on a one-million-document index, warm BM25 queries return in the microsecond range, with benchmarks in the benches directory. It offers multi-modal queries, so keyword, vector, and SQL queries run over the same rows, which gives agents flexible retrieval paths. It is object-storage-native, with snapshot-isolated reads and atomic commits, and it uses an open format with no lock in: text and numeric data is stored as spec-compliant Parquet, so anything that reads Parquet can read your data. Bindings exist for Python, Node.js, and Rust, and the Rust API reference lives on docs.rs. The engine installs the mimalloc global allocator by default, with an option to turn it off when embedding in a process that already sets one. This combination targets teams that want search and retrieval without standing up a separate search service, since the data already lives in their object store. The speed per dollar goal is what separates it from an in-memory search service, since the data already lives in cheap object storage that the engine queries directly.

Querying across modes

infino exposes three query styles over the same rows. BM25 gives keyword search, vector search gives k-nearest-neighbor lookup, and SQL filters rows by column. The quickstart shows creating a table with an FTS index on a body column and a vector index on an embedding column, then calling bm25_search, vector_search, and query_sql against the same data. A vector search can be restricted with a pushdown filter on another column, so the ranking is bounded by a keyword condition. The SQL path uses `query_sql` to select rows by source or other fields. The README gives parallel examples in Python, Node.js, and Rust, including a 16-dimension toy embedding so the samples run as written. Because the indexes are wired into SQL execution as physical access paths, a text predicate can be answered from the FTS index without a full column scan, and superfiles that cannot match are never opened. Term blooms, value ranges, and vector centroids live in the manifest, so scalar, keyword, and vector signals prune through one shared layer. The result is one snapshot and one copy of the data for sparse, dense, and structured predicates, which removes the second system a team would otherwise sync and the client-side stitching between separate stores. Hybrid search fuses keyword and vector rankings inside one SQL query, so the client does not stitch results together after the fact, which removes a common source of bugs.

The Parquet open format

A superfile in infino is a spec-compliant Parquet file. The embedded BM25 and vector index regions are spliced ahead of a standard Parquet footer and pointed at by infino metadata entries that any conformant Parquet reader ignores. That means the columnar body opens in DuckDB, pandas, pyarrow, or DataFusion with no infino in the read path and no export step. The README shows querying superfiles with DuckDB's read_parquet directly. There is a one-directional caveat: standard tools read a superfile's columns with no export, but rewriting it through a generic Parquet writer produces valid Parquet that has silently dropped the embedded indexes, so it is no longer a superfile. The shortest end-to-end demo writes a corpus, runs BM25 plus vector plus SQL and hybrid retrieval, then reads the same file back with DuckDB and pyarrow. This openness matters for longevity. Your data is plain Parquet that outlives the engine, while the indexes are extra regions a Parquet reader skips. The design trades bidirectional round-tripping for the guarantee that the raw columns are always readable by ordinary tooling, which protects you from vendor lock in at the storage layer. The one-directional rewrite caveat means you should treat superfiles as the canonical form and avoid round-tripping them through a plain writer that drops the embedded indexes.

Stability and development

infino pins its public API to a cargo-public-api snapshot so any change to the re-exported surface is reviewed as a contract change in the same pull request. The versioning policy keeps 0.x while the surface soaks and targets 1.0 after a release or two without churn, with pre-1.0 breaks shown in the snapshot diff and called out in release notes. Growable public enums and structs are marked non-exhaustive so adding a variant is not a breaking change. The API is Arrow-native, so a major bump of arrow or datafusion that changes an exposed type is a breaking change to infino, and the supported range is documented and CI-tested. The minimum supported Rust version is 1.95, enforced by rust-version in Cargo.toml. The Python and Node packages version independently and each embed their own copy of the engine. Development uses a pinned rust-toolchain.toml, `cargo test --features test-helpers` for the suite, and `make ci` before a pull request. Memory safety is checked with miri and AddressSanitizer. Benchmarks live under benches and use a custom harness so build, correctness, hot reads, cold object-store reads, RSS, and markdown output share one measured lifecycle, which keeps reported numbers tied to a single run. The cargo-public-api snapshot makes the supported surface explicit, so a bump that changes a type is reviewed as a contract change rather than slipping through unnoticed.

Editorial conclusion

infino is written in Rust, distributed under the Apache-2.0 license, and its repository was last updated on 2026-08-24.