mlx-serve: a Zig inference server that speaks OpenAI, Anthropic and Ollama on Apple Silicon
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Includes MLX Core macOS app with chat, agent mode, and tool calling.
At a glance
- What is it?
- mlx-serve is a native Zig server for local LLM inference on Apple Silicon that exposes OpenAI, Anthropic and Ollama-compatible APIs on one port, with an optional macOS menu-bar app called MLX Core. The interesting part is the API surface and the no-Python build; the unverifiable part is the performance table the README puts front and centre.
- Who is it for?
- Adopt mlx-serve if you already drive Claude Code, the OpenAI SDK, or an Ollama client and want those tools pointed at local weights on an Apple Silicon Mac without a Python runtime in the loop. Do not adopt it if you need Linux or CUDA, if you require a licence you can read in one line, or if your decision depends on the +26% decode claim, which the README asserts against LM Studio on identical MLX weights but does not substantiate in the supplied material.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Zig, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem mlx-serve targets: one local port for tools that expect a cloud wire format
Most local inference tools on macOS ask you to change something. You swap your client, or you accept a partial API, or you install a Python environment because the runtime is Python. mlx-serve takes the opposite position. It runs a server on http://localhost:11234 and claims that the same port works with Claude Code, the OpenAI SDK, Continue, Cursor, and Open WebUI. It also implements the Ollama API surface (/api/chat, /api/generate, /api/tags, /api/embed, /api/pull), so tools already wired to Ollama, including Raycast, Obsidian, Enchanted and the ollama-python and ollama-js clients, are meant to keep working unchanged. The audience is narrow and specific: developers on Apple Silicon who want local weights behind an interface their existing tooling already speaks. If you are not on a Mac, or you are happy to rewrite your client integration, the compatibility layer buys you nothing.
What actually runs: Zig on top of MLX and an embedded llama.cpp
The README describes a native Zig server that loads MLX-format models and every GGUF on Hugging Face, with llama.cpp embedded rather than run as a separate process. That is the architectural decision worth noting. Rather than one engine and a converter, mlx-serve ships two runtimes behind one HTTP surface, and the model name you pass decides which one loads. Text is not the only output: the same server is documented as generating images, video, music, speech with voice cloning, and 3D models, all described as native MLX. On top of the server sits MLX Core, a signed and notarized macOS menu-bar app that bundles the server binary. The README states the app runs the same binary as the CLI on the same port, so an OpenAI or Anthropic client can point at it while the app is open. Agent mode with MCP tool calling and a tool set the README counts at ten lives in the app layer. The README also mentions a sandboxed agent shell that runs in an isolated Linux VM, which is a notable claim about isolation and one I cannot verify from the supplied text.
Getting it running: Homebrew, the CLI, and the build path
The documented install path is Homebrew. You tap the repository and then choose either the app or the CLI:
brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve brew install --cask mlx-core brew install mlx-serve
From the CLI, the README gives an Ollama-shaped workflow. mlx-serve run gemma4 downloads Gemma 4 E4B at 4-bit, serves it and drops you into a terminal chat. mlx-serve pull qwen3.6:27b downloads without serving, and the README notes the pull is resumable and comes straight from Hugging Face. mlx-serve list shows what is on disk. mlx-serve serve loads models on demand by name. Short names, org/repo Hugging Face ids and name:tag forms are all accepted. Building from source is a different story: it needs Xcode 26.2 or newer with the Metal Toolchain component, and the README gives a check for it. If xcrun -sdk macosx metal --version fails, you run xcodebuild -downloadComponent MetalToolchain. Then git clone --recurse-submodules, brew bundle install --file=Brewfile for cmake and webp, and ./app/build.sh for an ad-hoc signed app plus server. The README states Zig, mlx and llama.cpp are pinned and fetched or built by that script, with no Python anywhere in the build. The runtime floor is macOS 26.2 or newer on Apple Silicon.
The performance table is the weakest part of the documentation
The README publishes a head-to-head table claiming a +26% decode geomean against LM Studio on identical MLX weights, with shipping defaults, and +11% against mlx-lm. It footnotes that Ollama cannot run MLX except for a handful of NVFP4 conversions, so that comparison is GGUF against GGUF, and the Ollama figure is marked as an estimate. The material supplied here contains no methodology, no hardware, no prompt set, no token counts. The README points to a Performance section that is truncated in what I can see. A number presented as a headline claim, with an estimate in the same column and no reproducible setup, is not evidence. Treat +26% as an assertion by the project, not a measurement you can rely on. The same caution applies to the feature matrix, which marks a long list of capabilities as present in mlx-serve and absent in three named alternatives; the matrix is the project describing itself, and its entries about other projects are not something this material lets me check.
Where it is the wrong tool, and what to compare against
The constraints are hard. macOS 26.2 or newer on Apple Silicon is a recent floor, so older Macs are out, and Intel Macs are out. There is no Linux or Windows path in the material. If your deployment target is a CUDA box or a container on a server, nothing here applies. The licence is the second problem. The repository metadata reports NOASSERTION, while the README carries an MIT badge and the comparison table lists the licence as MIT. Those two signals disagree, and the supplied material does not resolve which is correct. Anyone embedding this in a product needs to read the LICENSE file directly rather than trust either the badge or the metadata.
On alternatives, the meaningful contrast is with mlx-lm. mlx-lm is Apple's Python tooling: you get the reference MLX implementation and a Python runtime in the loop, and the README's own table marks it as requiring Python at runtime and lacking an OpenAI-compatible API. mlx-serve is the same family of weights behind an HTTP server with no Python in the build or at runtime. If your workflow is a Python notebook calling library functions, mlx-lm is the shorter path. If your workflow is a client that speaks HTTP, mlx-serve removes the glue code. Ollama is the other real comparison, and the difference is engine scope: Ollama is a Go server built around GGUF, while mlx-serve bundles llama.cpp for GGUF and adds a native MLX path alongside it, plus the Anthropic and Ollama API surfaces. LM Studio is proprietary and Electron-based per the README, and the README's own footnote concedes that recent LM Studio builds ship Anthropic /v1/messages and OpenAI /v1/responses endpoints with partial coverage, so the compatibility gap is narrower than the matrix implies.
Maintenance cost and the release cadence you are signing up for
The repository is not archived and the last push shown is 2026-09-10. The three most recent releases land close together: v26.9.2 on 2026-09-09, v26.9.1 on 2026-09-03, and v26.8.11 on 2026-08-29. Release titles reference per-model settings, chat providers, terminals in the sidebar, a 1M context figure, and a pinned MLX version (0.32.2 in v26.8.11). That cadence has a cost. The build pins Zig, mlx and llama.cpp, and the source build requires Xcode 26.2 or newer with the Metal Toolchain component, so a toolchain bump on Apple's side can break ./app/build.sh independently of anything the project does. If you build from source rather than installing the cask, budget for re-running the clone with --recurse-submodules and the Brewfile step when the pins move. If you install via Homebrew, brew upgrade handles the server, but the MLX version pin still moves under you, which matters if you have validated a specific model's output. The version scheme itself, year.month.patch, tells you the project intends to ship monthly at minimum.
What to verify before you point real work at it
Three things are checkable without trusting the README. First, the licence: open LICENSE and reconcile it against the MIT badge and the NOASSERTION metadata, because that determines whether you can ship anything built on this. Second, the API surface you actually need: start the server, point your existing client at http://localhost:11234, and exercise the specific endpoint. The README lists OpenAI, Anthropic, Ollama and OpenAI Responses with WebSockets, and it explicitly notes that mlx-serve implements the Responses WebSocket transport and /v1/responses/compact where LM Studio covers those surfaces only partially. Whether your client needs those is a question only your client can answer. Third, the build: run xcrun -sdk macosx metal --version first, and if it fails, xcodebuild -downloadComponent MetalToolchain before ./app/build.sh. If the source build is not something you want to own, the cask install avoids it, and the app and CLI share the same binary and port, so the choice does not change what your clients see.
Editorial conclusion
Adopt mlx-serve if you already drive Claude Code, the OpenAI SDK, or an Ollama client and want those tools pointed at local weights on an Apple Silicon Mac without a Python runtime in the loop. Do not adopt it if you need Linux or CUDA, if you require a licence you can read in one line, or if your decision depends on the +26% decode claim, which the README asserts against LM Studio on identical MLX weights but does not substantiate in the supplied material. Before committing, run ./app/build.sh from a clean clone and confirm the Metal toolchain check passes, then point an existing OpenAI or Anthropic client at http://localhost:11234 and exercise the one endpoint you actually depend on. The licence file is the first thing to read, because the repository metadata reports NOASSERTION while the README badge says MIT.
Community notes