Open-source alternatives to Cohere: 4 embeddings and reranking API tools compared
Cohere's embedding and reranking models are reached over a network, so replacing that API with self-hosted software means choosing a checkpoint you can pin, running the inference yourself and giving up an endpoint somebody else keeps current. Most readers should start with huggingface/text-embeddings-inference, the only candidate here that answers a request with a vector in the same shape Cohere does.
What Cohere does, and why people look elsewhere
Cohere is a hosted API for language models with a focus on search: it turns documents into embedding vectors and reranks retrieved passages before they reach a model.
- the documents you embed are sent to the vendor's servers
- the embedding and reranking models are the vendor's, and their output can change with a model update
- there is no edition of the service that runs inside your own network
The candidates at a glance
| Project | Licence | Maintenance | Language | GitHub stars | Install |
|---|---|---|---|---|---|
| huggingface/sentence-transformers | Apache-2.0Permissive: commercial use allowed | Commits in the last dayLast push October 23, 2026 | Python | 19,214 | Command documented |
| qdrant/fastembed | Apache-2.0Permissive: commercial use allowed | Commits in the last six monthsLast push October 15, 2026 | Python | 3,244 | Command documented |
| huggingface/text-embeddings-inference | Apache-2.0Permissive: commercial use allowed | Commits in the last six monthsLast push October 13, 2026 | Rust | 5,085 | See analysis |
| FlagOpen/FlagEmbedding | MITPermissive: commercial use allowed | Commits in the last six monthsLast push August 24, 2026 | Python | 12,212 | See analysis |
Each alternative in detail
huggingface/sentence-transformersState-of-the-Art Embeddings, Retrieval, and Reranking
- Best for
- You write Python, want a pretrained embedding or reranker running in a few lines inside your own service, and would like the option to finetune it later if the shipped checkpoint is not good enough.
- What you give up
- There is no HTTP service, so if your callers are not Python processes you have to build the endpoint yourself, and our analysis of Sentence Transformers warns that the transformers pin of 5.0.0 up to but not including 6.0.0 means a dependency bump can change scoring behaviour once a v6.0.0 model cache is in play. It also drags in PyTorch, so the runtime footprint is larger than the alternatives and the torch build has to match the CUDA or CPU target you deploy on.
- First install step
- Full setup guide
pip install -U sentence-transformers
qdrant/fastembedFast, Accurate, Lightweight Python library to make State of the Art Embedding
- Best for
- Your retrieval service has to start fast, run small or deploy to a place with no GPU, and you want one client library that can keep serving text, images, late-interaction models and rerankers as your needs grow.
- What you give up
- It runs on ONNX Runtime rather than a deep learning framework, so a checkpoint that relies on features the runtime does not implement is not available to you, and our analysis of FastEmbed says that adding a model the library does not ship is a supported path only if you name the pooling and the dimension yourself. The README also makes clear that the shipped catalogue is fixed, so a model change means waiting for a release.
- First install step
- Full setup guide
pip install fastembed # or with GPU support pip install fastembed-gpu
huggingface/text-embeddings-inferenceA blazing fast inference solution for text embeddings models
- Best for
- You serve embedding or sequence classification models to other services and need an HTTP or gRPC endpoint with token-based dynamic batching and low latency at a steady request volume.
- What you give up
- Model support is a fixed list, so our analysis of Text Embeddings Inference tells you not to adopt it if you need an architecture outside that list or custom pooling beyond the provided options, and it does not suit you if you want to avoid Docker and GPU dependencies. The Python SDK for it is not the interface you talk to, the endpoint is, and our analysis adds that you should test it against your expected batch size and sequence length.
FlagOpen/FlagEmbeddingRetrieval and Retrieval-augmented LLMs
- Best for
- You are committed to the BGE family and want dense, lexical and multi-vector retrieval from one toolkit, particularly in multilingual or multimodal work where the same rerankers serve both.
- What you give up
- Our analysis of FlagEmbedding says the repository mixes stable code with research experiments, so the feature you rely on may still be experimental in the release you pin, and it is the wrong choice if you want a single narrow API for English-only sentence embeddings. The toolkit also brings a model family rather than an API, so you decide the serving layer, the batching and the deployment yourself.
Four ways to serve an embedding: a server, a library, a light runtime, a model family
Cohere is a hosted API for language models with a focus on search. It turns documents into embedding vectors and reranks retrieved passages before they reach a model. The reasons teams look for something else are about control rather than quality: the documents you embed are sent to the vendor's servers, the embedding and reranking models belong to the vendor, and their output can change with a model update, so the vector for one text can differ between one month and the next. There is no edition of the service that runs inside your own network.
The four candidates here replace that endpoint in four different shapes.
huggingface/text-embeddings-inference is the closest to what a hosted API gives you. It is a Rust toolkit that serves embedding and sequence classification models, with token-based dynamic batching, inference code optimised through Flash Attention, Candle and cuBLASLt, safetensors and ONNX weight loading, and Prometheus metrics with distributed tracing. The README says it supports Nomic, BERT, CamemBERT, XLM-RoBERTa, JinaBERT, Mistral, Alibaba GTE, Qwen2, MPNet, ModernBERT, Qwen3 and Gemma3, and it carries sections for re-ranker models, for sequence classification and for air-gapped deployment.
huggingface/sentence-transformers is a library rather than a server. The README describes four ways in: a Sentence Transformer model for embeddings, a Cross-Encoder for similarity scores that doubles as a reranker, a Sparse Encoder for sparse embeddings, and a Multi-Vector Encoder for token-level embeddings in the ColBERT style. The same README covers training or finetuning your own embedding and reranker models.
qdrant/fastembed is a third option: a Python library that runs models on ONNX Runtime instead of a deep learning framework, so it needs no GPU and does not download large framework packages. Its README shows text embeddings with query and passage prefixes, image embeddings, late-interaction and late-interaction multimodal models, and rerankers.
FlagOpen/FlagEmbedding is the toolkit behind the BGE family and the widest in scope: dense, lexical and multi-vector retrieval, with the README listing BGE-VL multimodal models released under MIT.
Who was left out, and why the hosted rest of the market is not here
Three groups of tools are missing from this page, and each is missing for a reason worth stating.
The first is the hosted embedding providers other than Cohere. They are hosted in exactly the same way, with the same property you are trying to remove, so listing them would answer a different question.
The second is the vector databases. Qdrant, Milvus, Weaviate and the rest store vectors and retrieve them, but they do not decide what the vector means. A vector store without an embedding model in front of it has nothing to store until you have already solved the problem this page is about.
The third is the general-purpose inference servers, which can serve embedding models but are built for text generation and bring a heavier operational profile to a job that does not need one.
What is also absent is a reranker that does not arrive bundled with something bigger. If you want to reorder the top passages before they reach your model, the candidates here cover it in two ways: the Cross-Encoder class in sentence-transformers and the reranker support in fastembed for Python code, and the re-ranker model support in text-embeddings-inference for a served endpoint. The BGE family also ships reranker checkpoints, and FlagEmbedding is the toolkit that loads them.
Choosing by scenario: one developer, a small team, a company with residency rules, a developer embedding it in a product
A solo developer with a small corpus should look at huggingface/sentence-transformers. The README shows a model loaded by name, texts encoded into arrays, a similarity function over them, and a Cross-Encoder that scores query and passage pairs and can rank a list for you. Our analysis of Sentence Transformers says it is the default entry point for Python teams that need semantic search without training a model first, with the option to finetune later.
A team shipping an embedding service should look at huggingface/text-embeddings-inference. Our analysis of Text Embeddings Inference says it trades broad model support for speed and operational simplicity, and it fits when you serve supported models at scale and need low latency with token-based dynamic batching. The README notes small container images and fast boot times, and a section for running locally on a Mac with Metal support.
A company with data residency requirements can ask all four the same question: where does the model run, and where does the text go. For sentence-transformers, fastembed and FlagEmbedding the answer is inside your own process, and for text-embeddings-inference it is a container on a host you control. None of them sends your documents to a third party. What changes between them is the operational shape, not the answer.
A developer embedding retrieval inside a product should think about the upgrade path rather than the first result. Our analysis of Sentence Transformers points at the transformers pin and notes that a v6.0.0 model cache can change scoring behaviour, so a dependency bump can move your rankings without a code change. Our analysis of FlagEmbedding makes a related point: the repository mixes stable code with research experiments, so the feature you rely on may still be experimental in the release you pin.
What each one costs in operations: a process, a container, or a GPU
The four shapes carry four different bills.
sentence-transformers is a library inside your own service. The README recommends Python 3.10 or later, PyTorch 2.2 or later and transformers 5.0 or later, and the install command on this page is a single pip install. The cost is inside your process: our analysis says to confirm that your torch build matches the CUDA or CPU target you actually deploy on. On CPU, the latency of a large model is a product decision rather than a library decision.
fastembed is a library with a smaller appetite. The README says it needs no GPU and does not download gigabytes of framework packages because it uses ONNX Runtime, and it names AWS Lambda as a target runtime. Our analysis of FastEmbed says it fits a retrieval service that has to start fast or run small, and it warns of two practical details: read the version matrix in the packaging file, because specific inference runtime versions are excluded per Python version, and decide early whether you need models the library does not ship, since adding one means naming the pooling and the dimension yourself.
text-embeddings-inference is the one you operate as a service. It is distributed as container images, and the README carries a local install option, a Docker build section and Apple Silicon support through Homebrew. Our analysis says it does not suit you if you want to avoid Docker and GPU dependencies, and that you should test the endpoint against your expected batch size and sequence length before you rely on it.
FlagEmbedding sits between the library and the platform: it is a Python toolkit that runs the BGE models, and it brings the research side of the repository with it. Our analysis says to check the model card for input length limits and language coverage, and to review the research directory to see whether the feature you rely on is still experimental.
Licences: permissive code, and the question that is actually about the weights
All four are permissive, and that is worth saying plainly. huggingface/sentence-transformers is Apache-2.0, qdrant/fastembed is Apache-2.0, huggingface/text-embeddings-inference is Apache-2.0 and FlagOpen/FlagEmbedding is MIT. None of them asks you to publish changes if you run a modified copy inside your own network, so the licence conversation ends quickly for a company standardising on one of them.
The question that is not about the code is the weights. A library licence covers the code; the checkpoint you load carries its own terms, and the README for text-embeddings-inference has a section for using a private or gated model, which is the mechanism for a model you cannot simply download. Our analysis of Sentence Transformers tells you to check that the specific checkpoint you plan to ship is listed on the Hugging Face models page filtered by the library, which is a practical way to confirm the model exists in the form you expect.
One project makes the weight licensing explicit. The FlagEmbedding README announces BGE-VL multimodal embedding models released under MIT and free for academic and commercial use, and it lists reranker checkpoints that are lighter than their predecessors because they use token compression and layerwise operations. That is the family to read when the question is what you may ship.
The practical consequence for a team replacing a hosted endpoint is that you now own the version. A hosted service can change the vector for a document when the vendor ships a model update; a pinned checkpoint does not, and you decide when to re-index.
Bottom line
Take huggingface/sentence-transformers when a Python library inside your own service is enough, huggingface/text-embeddings-inference when you need an endpoint with dynamic batching, qdrant/fastembed when you have no GPU and want the process to start fast, and FlagOpen/FlagEmbedding when you are committed to the BGE family and want its rerankers. Before committing, pin the checkpoint and read its model card, confirm the torch build or the container base matches the hardware you actually deploy on, and check whether your index survives a library version bump.
Sources
- Cohere official site
- huggingface/sentence-transformers repository
- huggingface/sentence-transformers README
- qdrant/fastembed repository
- qdrant/fastembed README
- huggingface/text-embeddings-inference repository
- huggingface/text-embeddings-inference README
- FlagOpen/FlagEmbedding repository
- FlagOpen/FlagEmbedding README
Was this useful?