fastembed-rs: Local ONNX Embeddings and Reranking in Synchronous Rust
Rust library for generating vector embeddings and reranking locally!
At a glance
- What is it?
- fastembed-rs wraps ONNX Runtime and Hugging Face tokenizers behind a synchronous Rust API for text, sparse, image and reranker models. It is a good fit for Rust services that want embeddings without a Python sidecar, and a poor fit if you need GPU serving or an async-native client.
- Who is it for?
- Adopt fastembed-rs if you already write Rust and want embeddings or reranking inside the same process, with no Python service and no Tokio requirement. Skip it if you need an async client, GPU execution, or model training and fine-tuning, since the library is an inference wrapper and nothing more.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 3 days ago.
- What is it written in?
- Mainly Rust, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap fastembed-rs fills: embeddings without a Python process
Most retrieval stacks in 2026 still compute embeddings in Python. That means a separate process, a separate dependency tree, and a serialization boundary between your service and its vectors. fastembed-rs removes that boundary for Rust codebases. The README describes it as a "Rust library for generating vector embeddings, reranking locally", and the feature list makes the intent explicit: synchronous usage with no dependency on Tokio, ONNX inference through pykeio/ort, and tokenization through huggingface/tokenizers. The audience is narrow and clear. If your service is written in Rust and you want to embed documents, embed queries, or rerank a candidate list inside the same binary, this library is aimed at you. If you are building in Python, Go or JavaScript, the README points to sibling projects (fastembed, fastembed-go, fastembed-js) rather than pretending to serve you. The project is licensed Apache-2.0.
What actually happens between a string and a vector
The architecture is visible from the dependency choices. A caller constructs a model object such as TextEmbedding, passing either Default::default() or a TextInitOptions value that names the model and tuning parameters. Tokenization is delegated to the Hugging Face tokenizers crate, so the same tokenizer files that ship alongside the ONNX weights are used at inference time. The token ids then go through ort, the Rust binding to ONNX Runtime, which executes the model graph. The output is the vector your caller receives. Reranking follows the same shape but produces relevance scores for query and document pairs rather than embeddings. Sparse embedding models such as prithivida/Splade_PP_en_v1 and BAAI/bge-m3 produce sparse vectors instead of dense ones, which matters if your index is built for learned sparse retrieval rather than cosine similarity. Image embedding runs a separate path with its own default, Qdrant/clip-ViT-B-32-vision. The README also notes that nomic-ai/nomic-embed-text-v1.5 pairs with nomic-embed-vision-v1.5 for image-to-text search, and Qdrant/clip-ViT-B-32-text pairs with clip-ViT-B-32-vision, which tells you the text and vision towers are exposed as distinct model entries that a caller must align manually.
Supported models and the backend split you have to plan for
The model list is long and unevenly supported. Text embedding covers the BGE family in English and Chinese, several sentence-transformers models, the nomic-embed-text line, multilingual E5, mxbai-embed-large-v1, GTE, ModernBERT-embed-large, Jina code and English embeddings, Google embeddinggemma-300m, snowflake-arctic-embed in five sizes, and the Qwen3-Embedding models at 0.6B, 4B and 8B parameters. The default is BAAI/bge-small-en-v1.5. Quantized variants exist for several models, selected by appending Q to the enum variant, for example EmbeddingModel::BGESmallENV15Q, and EmbeddingGemma has a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4. This is where the design gets less uniform. The README states that nomic-ai/nomic-embed-text-v2-moe requires a nomic-v2-moe feature and that the Qwen3 embedding models require a qwen3 feature, both on the candle backend, while the general inference path is ort. Qwen/Qwen3-VL-Embedding-2B is multimodal and exposed through a separate Qwen3VLEmbedding type. So the library is not one runtime with one model list. It is at least two backends, and the largest and most recent models sit on the one that is not the default. Reranking is a much shorter list: bge-reranker-base as default, bge-reranker-v2-m3, jina-reranker-v1-turbo-en, and jina-reranker-v2-base-multilingual.
Getting it into a Cargo project
Installation is a single command, per the README: cargo add fastembed. The alternative is adding the dependency by hand. The README shows fastembed = "5" in Cargo.toml, though the most recent release listed for the repository is v6.0.3, so the snippet in the README lags the release line and you should check crates.io for the current version rather than copying the line verbatim. Construction is where the configuration lives. The README gives two forms: TextEmbedding::try_new(Default::default()) for the default model, and TextEmbedding::try_new(TextInitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true).with_intra_threads(4)) for an explicit model with download progress reporting and a thread count. The with_intra_threads(4) call is the only performance-related knob shown in the material; the README does not document a broader set of tuning options, so treat thread count as the visible surface rather than the whole configuration story. The example document list also shows a convention worth copying: prefixes such as "passage: " and "query: " are used in the sample strings, with the comment that you can leave out the prefix but it is recommended. That recommendation is model-specific in practice, and the README does not map prefixes to models, so you have to check each model card on Hugging Face.
Where fastembed-rs is the wrong tool
The synchronous design is a deliberate trade-off and it cuts both ways. No Tokio dependency is a benefit for a CLI, a batch job, or a thread-pool service. It is a liability if your application is already async and you expect to await an embed call. Nothing in the material suggests an async API, so an async service will need to move embedding onto a blocking thread pool itself, and the library will not do that for you. The backend split is the second constraint. A model that requires the qwen3 or nomic-v2-moe feature runs on candle rather than ort, which means the largest models in the list do not share the default execution path. If you standardize on the default build and later pick Qwen3-Embedding-8B, you are changing backends, not just a model name. Third, this is an inference wrapper. There is no training, no fine-tuning, and no evaluation harness in the material. If your task needs a model adapted to your domain, fastembed-rs only serves the artifact someone else produced. Finally, the README does not state which ONNX Runtime binaries are bundled or how they are resolved on each platform, so cross-compilation and deployment on unusual targets are things you must verify yourself rather than assume.
Alternatives and the real difference in approach
The most direct alternative is the Python fastembed library from Qdrant, which the README itself lists under "Not looking for Rust?". The difference is not only language. The Python package gives you the same local-inference idea, but it runs inside a Python process with the Python ecosystem around it, which is the right choice if your retrieval pipeline is already Python and your service boundary is already a network call. Choosing fastembed-rs is choosing to delete that process boundary, and you pay for it by giving up Python tooling. A second alternative is calling a hosted embedding API. That inverts the trade: no model files to download, no ONNX Runtime to link, but your text leaves your infrastructure and you inherit per-token cost and network latency. fastembed-rs is the local option by definition, so this is a data-residency and cost decision more than a technical one. A third path is using ort directly without fastembed-rs. That gives you full control over model loading and execution, at the cost of writing your own tokenizer wiring and model registry. fastembed-rs is essentially that wiring, preassembled, with a fixed model enum. If your model is not in the list, the library will not help you, and ort directly will.
Maintenance cost, version drift and the licence
The repository is not archived, the last push recorded is 2026-09-07, and the release cadence is quick: v6.0.1 on 2026-08-23, v6.0.2 on 2026-08-27, v6.0.3 on 2026-09-07. Three patch releases in roughly two weeks is a signal to pin an exact version rather than a caret range, because a minor or patch bump can move model enum variants or backend features. The README's own Cargo.toml snippet showing fastembed = "5" while the release line is at 6.x is a concrete example of documentation lag, and it means you should read the changelog between versions instead of assuming the README matches the crate. Upstream, the README asks users to donate to ort, the ONNX Runtime wrapper, describing it as the primary upstream dependency. That is an honest statement of where the maintenance burden sits: fastembed-rs is a layer over ort and huggingface/tokenizers, so its stability is bounded by theirs. On licensing, the project is Apache-2.0, which permits commercial use and modification. That covers the library code. It does not automatically cover the model weights, which are hosted on Hugging Face under their own terms, so if you ship a product built on a specific model, check that model's licence separately. This is a description of the stated licence, not legal advice.
Editorial conclusion
Adopt fastembed-rs if you already write Rust and want embeddings or reranking inside the same process, with no Python service and no Tokio requirement. Skip it if you need an async client, GPU execution, or model training and fine-tuning, since the library is an inference wrapper and nothing more. Before committing, verify which backend your chosen model needs: the README marks nomic-embed-text-v2-moe and the Qwen3 embedding models as requiring the nomic-v2-moe or qwen3 feature on the candle backend, while the default text models run through ort. Also confirm the version you pin, because the README shows fastembed = "5" in Cargo.toml while the latest release listed is v6.0.3.
Community notes