mlx-embeddings: text, image and token embeddings on Apple silicon through MLX

What mlx-embeddings is for, and who ends up using it

Retrieval pipelines usually need an embedding step somewhere. On a Mac, the default answer has been to call a hosted embedding endpoint or run a model through PyTorch on the CPU. The first option sends text and images off the machine. The second works, but encoder inference on CPU is slow enough that batch jobs become overnight jobs. mlx-embeddings targets that gap: it loads encoder checkpoints into MLX, Apple's array framework for Apple silicon, and exposes them through a small Python surface. The README states the package is for running vision and language embedding models locally on a Mac using MLX. The topics on the repository (rag, retrieval-augmented-generation, embeddings) point at the same audience: people building local retrieval, semantic search, or classification who do not want an API dependency in the loop. It is a library, not a service. There is no server, no index, no storage layer. You get arrays back and you decide what to do with them.

Two loading paths, and why they do not feel like one package

The README documents two entry points that behave differently. The older path is from mlx_embeddings.utils import load, which returns a model and a tokenizer. You encode text yourself with tokenizer.encode(text, return_tensors="mlx"), call the model, and read either outputs.last_hidden_state[:, 0, :] for the CLS token or outputs.text_embeds for pooled embeddings. The newer path is from mlx_embeddings import load, used with Qwen3-VL, which returns a model and a processor and hands the work to model.process(inputs, processor=processor). That second path accepts a list of dictionaries, where each item carries a text key, an image key, or both, plus an optional instruction string. The reranking variant takes a single dictionary with query and documents keys and returns a score per document rather than a vector. This is two APIs in one distribution. Code written against the tokenizer path will not transfer to the process() path, and the import location differs (mlx_embeddings.utils versus mlx_embeddings). If you are choosing a checkpoint, that choice determines which style you write against.

Architecture coverage is unusually wide for a package this size

The supported list spans XLM-RoBERTa, BERT, ModernBERT, Qwen3, Qwen3-VL, Llama-based bidirectional models such as NVIDIA NV-Embed, Llama Nemotron VL, and openai/privacy-filter. That last one is not an embedding model in the usual sense. The README describes it as a bidirectional 1.5B-parameter, 50M-active sparse mixture-of-experts token classifier that tags personally identifiable information with BIOES spans across eight categories: person, email, phone, URL, address, date, account number, and secret. So the same package covers sentence vectors, cross-modal vectors, reranking scores, sequence classification (the README shows NousResearch/Minos-v1 with model.config.id2label), masked language modeling, and token-level PII spans. That range is the strongest argument for the package. It also means the abstraction is thin: each task reads a different output attribute (text_embeds, pooler_output, logits) and applies its own post-processing. The library gives you the weights and the forward pass, not a task framework.

Getting it running, and the pooling detail that changes your vectors

Installation is one line: pip install mlx-embeddings. A minimal text embedding run, taken from the README, loads mlx-community/all-MiniLM-L6-v2-4bit through mlx_embeddings.utils.load, encodes a string with return_tensors="mlx", and reads outputs.text_embeds. The README notes that text_embeds use mean pooling for BERT and XLM-RoBERTa, while for ModernBERT the pooling strategy is set through the config file and defaults to mean. That is worth pausing on. If you swap a BERT checkpoint for a ModernBERT one and the config specifies something other than mean pooling, your vectors change meaning without any code change on your side. For the Qwen3-VL path, the README example builds four input items (text only, text only, image only, and text plus image), calls model.process, multiplies the result by its transpose for a similarity matrix, and prints a shape of (4, 2048). The reranker example prints a shape of (3,) for three documents. Note that mx.eval is called explicitly in both examples, which is how MLX defers computation until you ask for it.

Where it stops being the right tool

The package is Mac-only by construction. MLX is Apple's framework for Apple silicon, so there is no CUDA path and no Linux deployment story in the material provided. If your retrieval stack runs in a container on x86, this is not a component you can drop in. The second limit is the API surface. Two load functions in two modules, two calling conventions, and task-specific output attributes mean that upgrading across versions carries real breakage risk, and the release history is thin: v0.0.4 in September 2025, v0.0.5 in October 2025, then v0.1.0 in March 2026. A 0.1.0 version number after more than a year of releases tells you the maintainer is not treating the API as frozen. Third, the README does not document batching semantics, memory behaviour, or throughput for the tokenizer path, so anyone planning large offline jobs is working without published guidance. Finally, the README's own feature list is modest (single-item and batch processing, similarity utilities) while the model list is long. The gap between those two suggests the model support is ahead of the ergonomics.

The licence metadata does not match the README

The repository metadata reports the licence as NOASSERTION, meaning GitHub could not identify a standard licence from the files present. The README, in the same document, states "Free software: GNU General Public License v3" and the badge line points at the PyPI package. These two statements are not the same, and the discrepancy matters more than usual here. GPL v3 is a copyleft licence. If the README is accurate, distributing a product that links this library pulls the usual copyleft obligations into scope, which is a very different proposition from a permissive licence for anyone embedding it in a closed-source application. I cannot resolve the conflict from the material available, and I am not giving legal advice. The practical step is to read the actual LICENSE file in the repository rather than the README line or the GitHub badge, and if the answer determines your architecture, get it reviewed. Treat the README's GPL v3 claim as the working assumption until the file says otherwise.

What you would use instead, and the actual difference

sentence-transformers is the obvious comparison point, and the difference is not just speed. sentence-transformers runs on PyTorch, so it works on CUDA, CPU, and Apple silicon via MPS, and it ships a much larger wrapper layer: pooling configuration, similarity utilities, and a training API are part of the package rather than something you assemble. The cost is that its Mac path goes through MPS rather than a framework built specifically for unified memory, and it drags in the PyTorch dependency tree. mlx-embeddings inverts both of those. It is narrower (inference and a handful of utilities, no training), it is tied to one vendor's hardware, and in exchange the array operations target MLX directly. If your constraint is portability across machines, sentence-transformers wins on that axis alone. If your constraint is keeping inference on a Mac with minimal dependencies, the trade runs the other way. The Qwen3-VL multimodal and reranking path is the part with no direct equivalent in the sentence-transformers surface described in the README.

Maintenance cost and what to verify before you depend on it

Maintenance here is mostly about pinning. The package moved from 0.0.5 to 0.1.0 between October 2025 and March 2026, and the last push to the repository was in May 2026. With a pre-1.0 version and two calling conventions in the README, a loose version spec in requirements.txt is a liability. Pin the exact version you validated. The second cost is checkpoint drift: the README's examples reference mlx-community/all-MiniLM-L6-v2-4bit, mlx-community/answerdotai-ModernBERT-base-4bit, NousResearch/Minos-v1, openai/privacy-filter, and two Qwen3-VL checkpoints. Quantized community checkpoints can be re-uploaded or replaced, so record the revision you pulled, not just the name. The third is the pooling default noted earlier: if you switch encoder families, re-check the config rather than assuming text_embeds means the same thing. On licence, the NOASSERTION metadata against a GPL v3 README line is the item I would resolve first, because it constrains what you can ship regardless of how well the library performs.

Editorial conclusion

Adopt mlx-embeddings if you are on Apple silicon and want embedding inference to stay on the machine, and you accept that the API surface differs between the Qwen3-VL path and the older load() path. Do not adopt it if you need a permissively licensed dependency, GPU portability, or a stable public API across releases. Before committing, verify the actual licence file against the GPL v3 claim in the README, confirm which pooling strategy your chosen checkpoint resolves to, and pin a version, since 0.1.0 landed in March 2026 and the previous release was five months earlier.

Official sources

1. Blaizzy/mlx-embeddings on GitHub
2. Issues
3. README
4. Releases