Model or dataset
raullenchai/Rapid-MLX avatar
raullenchai/Rapid-MLX

Rapid-MLX publishes its own counter numbers next to the headline

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

4,152 stars443 forksPythonNOASSERTION

At a glance

What is it?
Rapid-MLX is an OpenAI- and Anthropic-compatible inference server and desktop app for Apple Silicon built on MLX, aimed at coding agents that need tool calls to work. Its comparison table is unusually candid: it dates the facts, marks unverified cells with a dash, and the measured section states where the engine is slower as well as where it is faster.
Who is it for?
Rapid-MLX suits a Mac developer whose agent cannot live without working tool calls and who wants a server that speaks both major API shapes on one port. It does not suit anyone benchmarking on a single stream or on cold prefill, since the project itself says those cases are weaker.
Can I use it commercially?
Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
Is it still maintained?
Yes. The repository received new commits within the last day.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • raullenchai/Rapid-MLX4,152 ★ · +430
Star history
Repository2026-09-102026-11-15
raullenchai/Rapid-MLX3.7k4.2k

430 Stars in the last 66 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

The repository description and the README disagree on the numbers

The short description at the top of the repository makes a set of claims that the README itself contradicts. The description says 4.2 times faster than Ollama and 17 tool parsers. The README states a measured 3.0 times aggregate decode throughput against Ollama, and the comparison table counts 27 parser modules. Neither number in the description is used anywhere in the document. The README's own version of the claim is more careful in every direction: it names the model, the machine, the concurrency level, and it links a page with the method, the raw data and a section on where it is slower. So a reader who takes the repository blurb at face value is quoting numbers the project itself has replaced, which is worth knowing before you paste them into anything.

The measured number comes with its own footnotes

The measured paragraph is short and dense, and every clause narrows the claim. The 3.0 times figure is 82.9 against 27.2 tokens per second of aggregate decode, on a 32 GB M2 Pro Mac mini, comparing version 0.12.11 of this engine against version 0.32.7 of the other one, at eight concurrent streams on a mixture model. Then the caveats, in the project's own order: including prefill, whole batch throughput was 1.6 times; single stream decode was about 1.5 times; a dense 12B model was no faster at all; and engines in the llama.cpp family prefilled cold prompts faster. A separate page covers a head to head against two other projects on two other machines, covering decode, agent sessions, restart, concurrency, and again where this engine is slower, with raw data attached.

The comparison table dates its facts and marks gaps with a dash

The table puts this project next to four others and does two things that documentation usually avoids. First, it states the date the facts were taken, which is late September 2026, and says they come mostly from each project's own documentation rather than from measurement. Second, it defines its own missing value: a dash means not verified for this table, and the footer invites readers to open an issue if they spot an error. The local API row is the one that decides compatibility for most people, and here this project exposes three endpoints, a chat completions path, a responses path and an Anthropic messages path. The engine row is close to uniform: three of the five use MLX on a Mac, one pairs a GGML engine with an optional MLX engine for models in safetensors form, and one offers two engines.

The prompt cache is a radix prefix and it survives a restart

The cache row is where the four projects genuinely differ, and this one has the longest cell in the table. What it uses is a radix prefix cache held in memory, with state snapshots for models that are hybrids, and it is written to disk on shutdown and restored at startup, alongside a quantized KV cache. That last clause is the one that matters for an agent workflow: a cache that survives the process means the first request after a restart is not paying full prefill. The alternatives are a tiered cache with a hot tier in memory and a cold tier on SSD reused across restarts, a cache that reuses the previous request's prompt, a cell that is simply not verified for one of the products, and a plain in-memory prompt cache with a switch for quantizing the KV cache on the command line.

Concurrency is the row where a default of one shows up

Four of the five projects are described as doing continuous batching, and the fifth is not: the reference implementation serves one request at a time while quantizing its KV cache, even though it uses the same batching generator internally for other paths. The product that ships to end users runs requests in parallel per model instead of batching, and the table gives the environment variable that controls it with a default of one, which means a user who has not set it is not getting concurrency at all. One other product moved to continuous batching for its MLX engine in a versioned release, which is the kind of detail that dates a table. So if you are choosing on throughput for an agent that fires parallel tool calls, this row matters more than the decode numbers.

Tool call parsing is where the module count lives

The project is described as focused on reliable tool calling for coding agents, and the parsing row is the evidence. This project has 27 parser modules, including an automatic detection fallback, living in one directory in the source tree. That number is the second place where the repository description is out of date, since it claims 17. The other projects take three different approaches: one uses the formats the underlying library already supports plus the families its own readme lists, one documents tool calling as a first class capability, one exposes tool use through the compatible API, and the reference implementation only offers tool calls for models whose tokenizer declares support for them. So a model that works in one of the others may need an explicit parser here, which is the trade the count pays for.

The Makefile spends more lines picking an interpreter than running tests

The developer target file is more interesting than the list of targets suggests, because of the comment above the interpreter variable. It explains three fallbacks. An active virtual environment wins, so a contributor on an older interpreter gets their own rather than whatever is on the path. Failing that it walks a list of versioned interpreters and actually runs each one to check its version, because version manager shims appear on the path for every version even when only one is installed. And the comment notes that the bare python on macOS is often a system version below the declared requirement, which is why there is a last-resort fallback that lets you see a clean error instead of a traceback. The help output then lists targets with timings, from a lint run measured in seconds to a ten minute agent soak test that needs a server.

The dependency floor exists because of one crash

The manifest carries a long comment inside its dependency list, and it explains a version floor in terms of a bug. The floor was raised to a specific release of the MLX library and the matching release of the language model wrapper for an issue in the tracker, and the comment names the test that carries the empirical status of the cross-thread stream crash on that floor. What makes it worth reading is the second part: a compatibility shim for the default stream is described as deliberately retained even though the newer library release introduced thread-local streams and the wrapper switched to them, because a reproduction still crashed on the new floor. So the floor is not cargo cult, it is a documented crash with a documented workaround, and the release scripts and benchmark directories at the root are part of the same habit of keeping evidence next to the code.

Editorial conclusion

Rapid-MLX suits a Mac developer whose agent cannot live without working tool calls and who wants a server that speaks both major API shapes on one port. It does not suit anyone benchmarking on a single stream or on cold prefill, since the project itself says those cases are weaker. Before adopting, read the measured section rather than the headline, check the compatibility matrix for the agent you actually run, and read the dependency floor comment in the manifest before pinning versions, since it exists because of a specific crash.

Frequently asked questions

What is Rapid-MLX?

An open-source inference server and Mac app for Apple Silicon, built on MLX, that exposes OpenAI-compatible chat completions and responses endpoints plus an Anthropic messages endpoint. The package is on PyPI and installable through a Homebrew formula, and the project describes itself as focused on reliable tool calling for coding agents.

How is Rapid-MLX faster than Ollama?

The project's own measured figure is 3.0 times aggregate decode throughput, which is 82.9 against 27.2 tokens per second at eight concurrent streams on a mixture model, with the same caveats stated: 1.6 times including prefill, about 1.5 times single stream, no gain on a dense 12B model, and slower cold prefill than llama.cpp family engines.

How does Rapid-MLX differ from mlx-lm's server?

Both run on MLX. This project exposes three API surfaces including an Anthropic messages endpoint and ships a desktop app, while the reference server exposes an OpenAI-style chat API with no GUI, serves one request at a time despite using a batching generator internally, and keeps an in-memory prompt cache with a switch to quantize the KV cache.

Why does Rapid-MLX pin its MLX dependency versions?

The manifest comment says the floor was raised for an issue tracked in the repository, tied to a cross-thread stream crash, and that a default stream shim is deliberately retained because a reproduction still crashed on the new floor. The comment names the test file holding the empirical status.

Which coding agents are tested against Rapid-MLX?

Five agents in the top tier are exercised end to end on real weights before a release, and the project publishes a compatibility matrix with exact API coverage and setup status. Any client that accepts an OpenAI- or Anthropic-compatible endpoint can usually use it without an adapter, which covers most of the integrations listed in the readme.

Official sources

  1. Issues
  2. Project website
  3. raullenchai/Rapid-MLX on GitHub
  4. README
  5. Releases
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/raullenchai-rapid-mlx.svg)](https://hysenlabs.com/projects/raullenchai-rapid-mlx)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/raullenchai-rapid-mlx.svg)](https://hysenlabs.com/projects/raullenchai-rapid-mlx)

Was this useful?