EmbedAnything streams embeddings straight to a vector database from a pipeline with no PyTorch in it
Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀
At a glance
- What is it?
- A Rust workspace of four crates covering ingestion, inference and indexing, published both as a Rust library and as a Python package built with maturin. The central idea is vector streaming: preprocessing and model inference run on separate threads and results are written to the database as they are produced. The manifests disagree about the licence and the version, and the deprecation notice contradicts the example directly beneath it.
- Who is it for?
- EmbedAnything is aimed at one specific bottleneck: the cost of running embedding models, and the memory that accumulates while you wait. If your pipeline is stalling on inference rather than on retrieval, the vector streaming design is the reason to look, and dropping the PyTorch dependency is a real deployment win for anything running on a small machine or in a container.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Rust, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- StarlightSearch/EmbedAnything1,324 ★ · +18
| Repository | 2026-09-11 | 2026-11-15 |
|---|---|---|
| StarlightSearch/EmbedAnything | 1.3k | 1.3k |
18 Stars in the last 65 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
Four workspace members, one of which is also excluded from the workspace
The Rust side is a workspace of four crates, and the layout tells you the shape of the system.
One crate holds the processors, which is the ingestion side: reading documents, extracting text, chunking. One is the core library. One is the Python bindings. One is the server. Each has its own manifest and the workspace resolves them together.
The odd entry is the Python crate. It is listed as a member and also listed as excluded, with a comment explaining that the Python package has to be built by a different tool. That tool is maturin, and the Python project file configures it: the extension module feature is on, the release profile is selected, symbols are stripped, and the Python source directory and module name are both pinned explicitly.
So the Python package is not a wrapper around a prebuilt binary. It is the same Rust code, compiled by maturin into an extension module, with the module name written down in the build configuration so the import path does not drift.
Both build profiles enable runtime path searching. For a project that ships compiled extensions to Python, that is the setting that lets the shared libraries resolve after installation rather than only inside a build tree.
The licence is Apache in the Rust workspace and MIT in the Python classifiers
Two manifests in the same repository declare two different licences, and this is the kind of thing that matters if you intend to depend on it.
The Rust workspace package declares the Apache 2.0 licence, and that value is inherited by the crates in the workspace. The Python project file, for the same code, lists a classifier granting the MIT licence and points its licence field at a licence file rather than naming a licence at all.
Since the Python package is the compiled form of the Rust code, the two are not describing different components. They are describing the same component twice, differently. If you are checking licence compliance for a product you ship, the licence file is the thing to read, and the disagreement in the metadata is a reason to ask rather than a reason to assume.
The version numbers have a similar mismatch, though that one is explicable. The workspace declares a version that is one patch ahead of the newest published release, which is ordinary when the branch has moved on. The older release tags are also inconsistent in their formatting, with the newest carrying a prefix and the two before it not.
Together these are the details to check before you pin a version: what you install from a registry and what the source tree says about itself are not always the same statement.
Vector streaming is the actual idea, and it is a threading decision
Most of the feature list is table stakes for an embedding library. One entry is the design.
Vector streaming means separating document preprocessing from model inference. The stated reason is that embedding models are computationally expensive and slow, so a pipeline that does both in sequence spends most of its time with one resource idle. Splitting them turns a sequential bottleneck into a concurrent workflow, and the feature list states the mechanism plainly: file processing, indexing and inference run on different threads, which reduces latency.
The implementation detail that makes it work is in the explanatory section, and it is the part worth knowing. The embedding process runs separately from the main process, using multi-producer single-consumer channels in Rust to keep throughput up. And because embeddings are written directly into the vector database as they are produced, nothing accumulates in memory waiting for the batch to finish.
That last point is the difference between streaming and parallelising. If you run inference concurrently but still hold the results until the end, you have made the model faster and the memory worse. Writing through to the database as each chunk completes is what makes the memory profile flat.
The project links a blog post and a video explaining the pattern, which is fair given it is the part of the design you cannot infer from an API.
The deprecation notice and the example code contradict each other
There is a warning box, and then an example that does exactly what the warning says not to do.
The warning states that a model selection helper has been deprecated, and it appears twice in the readme with slightly different wording, once referring to the pretrained loader and once naming the loader itself. Both carry the same warning glyph.
Immediately below it is a Python example that begins by importing that helper by name from the package, alongside the model class and the configuration class.
import embed_anything
from embed_anything import EmbeddingModel, WhichModel, TextEmbedConfigSo the first three lines of the example demonstrate the deprecated call path.
Whether the import still works is not the interesting question. Whether it works is a compatibility detail that a patch release could settle either way. The interesting question is that the deprecation was announced in the same file, in the same section, in the next paragraph, and the sample was not updated.
For anyone using this as a reference, the practical rule is to read the signature of the loader in the source rather than copying the sample. The examples folder contains roughly two dozen other files, and given this one, they are worth treating as illustrations of shape rather than as code to paste.
ONNX support is narrower than the feature list implies, and the benchmarks cover one runtime
Two claims in the readme need to be read against each other.
The feature list describes the backends as supporting the Rust machine learning framework, the ONNX runtime and cloud models. The supported models section is more precise: any Hugging Face model works on the Rust framework, while ONNX runtime support is named for two specific model families. So the ONNX path is a narrower door than the feature list implies, and the two model families named are a good sign of which use cases it was built for, since one of them is a document model that returns image embeddings rather than text embeddings.
The benchmark section is honest about its limits in a way that is easy to miss. It states that it measures only embedding model inference speed, on one runtime, with a notebook link for the code. And then a line saying benchmarks against other frameworks are coming soon.
So there is exactly one number in the repository, on one runtime, with no comparison. The link to a hosted notebook means you can rerun it, which is better than a static table, but the comparison everyone actually wants is the one the readme says has not been done.
The listed model families give a sense of the intended surface: dense and sparse retrieval, a late interaction model, a re-ranker, a modern BERT variant, and a static embedding model that needs no transformer at inference.
The container pins one ONNX runtime version and installs numpy twice
The build file is where this project is least tidy, and it is worth reading if you build images from it.
The dependency layer is done properly. A cargo-chef stage computes a recipe from the manifests, a second stage cooks the dependencies once, and only then is the source copied in, so editing a source file does not invalidate the dependency cache. That is the standard way to keep a Rust image build fast.
Then the acceleration library is installed by hand: a signing key is fetched over plain HTTP from the vendor's package repository, decoded, added as an apt source, and the maths kernel development package is installed from it, with the library path pinned to a specific toolchain directory in the image. Then the Python build tool is installed with a flag that permits modifying a system Python installation.
Two details in the final stage are outright sloppy. The runtime is told to load its ONNX library from a path containing a specific version suffix that does not match the version the Python project pins as a dependency. And the dependency install list names the array library twice.
The default command runs the test suite rather than a server, and there are four separate image definitions at the repository root, none of which is obviously the one behind the prebuilt server image the readme recommends pulling.
The examples directory is the real feature list
Twenty-four entries at the top level of the examples folder describe what this library actually covers, in more detail than any table in the readme.
The modality coverage is visible in the filenames: plain text, text with optical character recognition, audio, images, a vision-language model, and a document model that embeds page images. There is a separate variant for running that document model through the ONNX runtime, which confirms that path is a first-class option rather than an afterthought.
The retrieval model families are covered the same way: a late interaction model in both native and ONNX form, a sparse model, a static embedding model, a general model family, a reranker and a reranker from another family, plus a hybrid search example that presumably combines the dense and sparse paths.
Then the mechanics: semantic chunking and late chunking as separate examples, ONNX model loading, direct import from a cloud storage bucket, and an adapters directory for vector databases.
Two directories are worth singling out. One is a search agent example described in the readme as using the index for reasoning, which is the clearest statement of intent in the repository: the index is meant to be consumed by an agent that searches, not only by a nearest neighbour lookup. The other is a notebooks directory, which suggests the longer examples live there rather than in the top-level files.
Editorial conclusion
EmbedAnything is aimed at one specific bottleneck: the cost of running embedding models, and the memory that accumulates while you wait. If your pipeline is stalling on inference rather than on retrieval, the vector streaming design is the reason to look, and dropping the PyTorch dependency is a real deployment win for anything running on a small machine or in a container. Two things to check first. Benchmark it yourself, because the published numbers cover inference speed on one runtime only and the readme says comparisons with other frameworks are still to come. And read the manifests rather than the badges, since the licence field and the version number in the workspace file do not match what the repository metadata and the newest release tag say.
Frequently asked questions
What is EmbedAnything?
A Rust project covering ingestion, inference and indexing for embeddings, published both as a Rust library and as a Python package built with maturin. It handles text, images, audio, PDFs and web pages, chunks them with built-in methods including semantic and late chunking, and streams the resulting embeddings into a vector database. Supported embedding families include dense, sparse, late interaction, re-ranking and a static model.
What is vector streaming in EmbedAnything?
It is the separation of document preprocessing from model inference onto different threads, using multi-producer single-consumer channels so the pipeline is concurrent rather than sequential. Because embeddings are written directly into the vector database as they are produced instead of being held until a batch finishes, memory stays flat while latency and throughput improve.
Does EmbedAnything require PyTorch?
No, and that is stated as a headline feature: no PyTorch dependency means a low memory footprint and easy deployment in the cloud. The backends are a Rust machine learning framework, the ONNX runtime and cloud models. Any Hugging Face model works on the Rust framework, while ONNX runtime support is named for two specific model families.
How do I use EmbedAnything with my vector database?
Through an adapter layer. The readme says you can pick any vector database adapter for retrieval-augmented generation, and it corrects its own claim downward, striking through one line of code and replacing it with one word. There is a dedicated adapters directory in the examples, and past collaborations named in the readme include two specific vector database projects.
How is the EmbedAnything Python package built and installed?
With maturin, which compiles the Rust crate as an extension module with the release profile and symbols stripped. The Python project pins the source directory, the crate manifest and the module name, requires Python 3.10 or newer, and declares exactly one dependency pinned to an exact ONNX runtime version. A prebuilt server image is also published, though the repository root holds four separate image definitions and the default command in one of them runs the test suite.
What licence is EmbedAnything under and what version is current?
The manifests disagree. The Rust workspace package declares the Apache 2.0 licence, while the Python project file lists a classifier granting the MIT licence and points its licence field at a licence file, so read that file if compliance matters. The workspace version is one patch ahead of the newest published release, and the older release tags are formatted inconsistently, with only the newest carrying a version prefix.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/starlightsearch-embedanything)Star history chart for your README
[](https://hysenlabs.com/projects/starlightsearch-embedanything)Was this useful?