LM Studio mlx-engine: Apple Silicon inference you can run from a terminal

The engine that ships inside LM Studio on Mac

The first substantive line of the README is the one that matters for expectations: LM Studio 0.3.4 and newer for Mac ships pre-bundled with mlx-engine. So this repository is not primarily a thing you install and run as a product. It is the engine, published so you can read it, patch it and drive it directly when the GUI is in the way.

GitHub reports Python as the primary language, and the tree confirms a real Python package rather than a wrapper script. `mlx_engine/` holds the engine itself, `demo.py` and `batched_demo.py` are runnable entry points, `demo-data/` ships images for the vision examples, and `tests/` sits alongside a `CONTRIBUTING.md`, a `ruff.toml` and a `.pre-commit-config.yaml`.

Three upstream projects do the actual work, and the README credits each with its licence: mlx-lm from Apple's mlx-explore for text inference, Outlines from dottxt-ai for structured output, and mlx-vlm from Blaizzy for vision.

The requirements file pins Python 3.11 for a reason

The prerequisites are stated with unusual precision, which is a good sign. You need macOS 14.0 Sonoma or greater, and you need `python3.11`. The README explains that requirements.txt is compiled specifically for python3.11, and that 3.11 is the Python version bundled inside the LM Studio MLX runtime. It suggests `brew install [email protected]` as a way to add it to your path without disturbing your default Python.

The setup is three commands:

git clone https://github.com/lmstudio-ai/mlx-engine.git
cd mlx-engine
python3.11 -m venv .venv
source .venv/bin/activate
pip install -U -r requirements.txt

Reading requirements.txt explains why the version matters. The pins are exact and current: `mlx==0.32.0`, `mlx-metal==0.32.0`, `transformers==5.15.0`, `torchvision==0.24.0`, `tokenizers==0.22.2`, `outlines-core==0.1.26`. Three of them are not PyPI pins at all but git references at specific commits: mlx-lm, mlx-vlm and outlines each point at a commit hash in a GitHub URL. That is reproducible today and is also the part most likely to need maintenance, since those commits are outside this repository's control.

Downloading a model with lms, then running demo.py

Models are fetched with LM Studio's own `lms` CLI rather than a Python downloader, which is a small but meaningful design choice: the same command that gets the file also puts it where the engine expects to find it.

The README's text example pulls a 4-bit Llama 3.1 8B Instruct and then runs the demo against it:

lms get mlx-community/Meta-Llama-3.1-8B-Instruct-4bit
python demo.py --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit

The README notes the download is 4.53 GB. Without a `--prompt` argument the script uses a default prompt, and adding one changes the output, as the Mistral example shows with `mlx-community/Mistral-Small-Instruct-2409-4bit`, listed at 12.52 GB. The size difference between those two examples is the most useful thing in the README for planning: a 4-bit 8B model is a normal evening's download, and a small instruct model at higher quality is a much larger one.

Vision models and speculative decoding both go through demo.py

The vision path passes local image files on the command line, and the README lists the supported families explicitly: Llama-3.2-Vision, Pixtral, Qwen2-VL and Llava-v1.6, each with its own `lms get` line.

python demo.py --model mlx-community/pixtral-12b-4bit --prompt "Compare these images" --images demo-data/chameleon.webp demo-data/toucan.jpeg

Speculative decoding uses the same script with a second model, a small draft model proposing tokens and the large one verifying them. The README's example pairs `mlx-community/Qwen2.5-7B-Instruct-4bit` with `lmstudio-community/Qwen2.5-0.5B-Instruct-MLX-8bit`, which is the shape of the trade: a much smaller model running fast to keep the large model supplied with tokens.

The prompt in that example is a chat template written out in full, with `<|im_start|>` and `<|im_end|>` markers and a system turn establishing the model's identity. Worth noting because it means the demo does no conversation management of its own; you supply the framing.

Contributing runs through pre-commit hooks

The development setup is short and specific. Before contributing, install pre-commit and then run it across all files, fixing whatever it reports before opening a pull request.

pip install pre-commit && pre-commit install
pre-commit run --all-files

The test section is equally direct: install pytest and run the suite from the repository root with `python -m pytest tests/`. To narrow it to one vision family, the README shows `python -m pytest tests/test_vision_models.py -k pixtral`, which implies the vision tests are parametrised by model and that some of them hit real models rather than running purely offline.

There is one attribution line that is easy to miss. The README states that Ernie 4.5 modelling code is sourced from Baidu, linking to the ERNIE-4.5-0.3B-PT tree on Hugging Face. Code lifted from a specific model release under those terms is a licensing consideration separate from the repository's own MIT licence, and it is the kind of note that is good practice to have written down.

No releases, which is the honest version number

This repository has no GitHub releases, and that is not an oversight. The README's own framing is that LM Studio bundles the engine, which means the version a user gets is the version inside an LM Studio build, not a tag on this repository. The real version markers live in requirements.txt at `mlx==0.32.0` and `mlx-metal==0.32.0`.

That has a practical consequence for anyone planning to depend on this. There is no changelog here, no support policy, and no statement about which Python versions or macOS versions future builds will target. If you are using the demo for learning or research, that costs you nothing. If you are considering the engine as a dependency in something you ship, the absence of releases is the thing to resolve first, because you would be pinning to a commit hash that only this repository's requirements file knows about.

Open issues sit at 92 on a repository with 133 forks. The README does not describe a triage process or a response commitment, so the practical reading is that this is an actively used codebase without a published support contract.

Editorial conclusion

mlx-engine is worth your attention if you want to understand what LM Studio's Mac backend is actually doing, because the demo path in this repository is short, the dependency pins are exact, and the model catalogue in the README names the four vision families it supports. Start with `lms get` plus `python demo.py` rather than the test suite, and pin to the Python 3.11 that the requirements file was built against, since that is the version LM Studio's own runtime uses. Two things to weigh before building on it: there are no GitHub releases, so nothing here is versioned independently of LM Studio's bundles, and several dependencies are git references to unpinned upstream branches rather than published versions. If you need a supported, released artefact rather than a bundled engine, this is the wrong dependency to pin a product on.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. lmstudio-ai/mlx-engine on GitHub
4. README