oMLX: A Menu Bar LLM Server That Puts KV Cache on SSD for Apple Silicon

The Problem: Local LLM Servers Force a Tradeoff Between Convenience and Control

The README frames the core pain as a choice between convenience and control. oMLX tries to deliver both by offering a menu bar app that manages a background server, plus a CLI that can start, stop, and restart that server. The server speaks the OpenAI-compatible API on localhost:8000, so any existing client can connect. That means you can point Claude Code or similar tools at it without custom integration code. The convenience comes from the app's auto-update and one-click setup. The control comes from environment variables and a settings file that persist across restarts.

Tiered KV Caching: How the SSD Cache Works

The tiered cache is the reason oMLX claims to make local LLMs practical for real coding work. Without it, a long session with a tool like Claude Code would either exhaust memory or force a full context reload. With it, the server can swap older tokens to disk and reload them when needed. The README does not specify the eviction policy or the size of the hot tier. That is a gap. You will need to experiment to see if the default behavior matches your memory constraints.

Getting It Running: DMG, Homebrew, or Source

Configuration is done through environment variables like `OMLX_MODEL_DIR` and `OMLX_PORT`, or by running `omlx serve --model-dir /your/path` once to persist settings to `~/.omlx/settings.json`. Logs go to two places: the service log under `$(brew --prefix)/var/log/omlx.log` and a structured server log at `~/.omlx/logs/server.log`. For MCP support, you can install the optional package: `/opt/homebrew/opt/omlx/libexec/bin/pip install mcp`. The source install requires macOS 15.0+, Python 3.11 to 3.13, and Apple Silicon. Those are hard requirements. If you are on an Intel Mac or an older macOS, this project will not run.

The Custom Kernel Trap: A 30x Performance Difference You Must Verify

The README provides a verification command: `python -c "from omlx.custom_kernels import native_kernel_status; print(native_kernel_status())"`. You should run this after any source or Homebrew install. If you skip this check, you might think your server is performing normally when it is actually running at a fraction of its potential. This is a concrete, project-specific gotcha that separates oMLX from servers that have a single uniform code path. The tradeoff is that the custom kernels are not optional for good performance on specific model families, but they add a build dependency that is easy to miss.

What It Supports: Models, Admin Dashboard, and Multi-Mac

There is also an experimental multi-Mac inference feature. Source builds can split a model across unequal-memory Macs using MLX pipeline ranks over Ring or Thunderbolt RDMA/JACCL. The README describes a Cluster dashboard with peer discovery, SSH verification, shard planning, and a live shard map. This is clearly early-stage. The README points to a separate document for setup and limitations, but that document is not included in the material I have. Treat this feature as a proof of concept, not a production capability. The complexity of coordinating two Macs over RDMA is high, and the README does not provide benchmarks or validation results.

Alternatives and Tradeoffs: Where oMLX Sits in the Local Inference Landscape

The README does not compare oMLX to Ollama directly, but the design choices make the difference clear. The continuous batching and tiered cache are features that Ollama does not advertise. The menu bar app is a user interface layer that Ollama lacks. The tradeoff is that oMLX is tied to macOS and Apple Silicon, while Ollama runs on multiple platforms. If you need cross-platform support, oMLX is the wrong tool. If you are all-in on Mac, the tiered cache could be the deciding factor.

Maintenance and Upgrade Costs: Auto-Update, Homebrew, and License

The macOS app includes in-app auto-update, so future upgrades are one click. Homebrew users can run `brew update && brew upgrade omlx`. The project is under Apache-2.0, which is permissive and allows commercial use, but you should read the license text yourself for any specific obligations. The README does not mention a contributor guide or a roadmap, so maintenance cost is unclear. The last push was in August 2026, and there are recent release candidates, so the project is actively maintained. However, the custom kernel requirement means upgrades could break if the Metal toolchain changes or if the precompiled kernels in the DMG lag behind the source. You should verify the kernel status after every upgrade. The log files are split into two locations, which is a minor operational overhead but not a burden. Overall, the upgrade path is straightforward for app users, but source builders face a recurring build dependency.

Editorial conclusion

Adopt oMLX if you are a Mac developer who needs a local, OpenAI-compatible server that keeps everyday models pinned in memory and swaps heavier ones on demand, with a menu bar UI and a CLI for automation. Do not adopt it if you are on Intel Macs, macOS versions before Sequoia, or if you need multi-GPU or non-Metal acceleration. Before relying on it, verify that your Python environment has the Metal toolchain if you build from source, and check `native_kernel_status()` to ensure custom kernels are active for GLM-5.2 or MiniMax M3, otherwise you will see a 30x slowdown. Also confirm your SSD has enough free space for the cold KV cache tier, as the README does not specify a limit.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes