mlx-swift-lm: Running LLMs and VLMs on Apple Silicon from Swift
LLMs and VLMs with MLX Swift
At a glance
- What is it?
- MLX Swift LM is a Swift package for loading and generating with LLMs and VLMs on top of MLX Swift, with fine-tuning, grammar-constrained decoding and a bridge into Apple's FoundationModels. It is a library for Swift developers who want local inference in an app, not a server runtime.
- Who is it for?
- Adopt mlx-swift-lm if you are shipping a Swift app on Apple Silicon and want local LLM or VLM inference without writing your own Metal kernels; the MLXHuggingFace macros plus LLMRegistry give you a working ChatSession in a few lines. Do not adopt it if you need a language-agnostic serving layer or run on non-Apple hardware, because the package is built on MLX Swift and Apple frameworks.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Swift, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap mlx-swift-lm fills for Swift apps
Running a language model inside a Swift application usually means choosing between a Python process you cannot ship and a C++ runtime you have to wrap by hand. MLX Swift LM sits in the middle: it is a Swift package that builds tools and applications with LLMs and vision language models on top of MLX Swift, which is the Apple array framework. The audience is Swift developers who want generation to happen on the device or on a Mac, inside the same process as the rest of the app.
The package is not a single monolith. The README lists separate documentation sets for MLXLLMCommon (the shared API for LLM and VLM), MLXLLM and MLXVLM (example model implementations), MLXEmbedders (encoders and embedding models), MLXGuidedGeneration (grammar-constrained output) and MLXFoundationModels. That split matters when you are estimating adoption cost, because you can take the common layer plus one model family and ignore the rest.
The stated feature set is narrow and concrete: model loading with integrations for several tokenizer and model downloading packages, LoRA and full fine-tuning including quantized models, and many model architectures for both LLMs and VLMs. There is no mention of a server, a REST API or a CLI in the material provided.
Protocol conformance is the integration mechanism
The design decision that shapes everything else is that this package does not ship its own tokenizer or downloader. According to the README, it integrates with a variety of tokenizer and downloader packages through protocol conformance, and users pick from three integration approaches that trade freedom against convenience. The documentation for that choice lives in the MLXLMCommon using guide.
The convenience path is the MLXHuggingFace macros, described as providing a default Hugging Face downloader and tokenizer integration. In the README example, `#huggingFaceLoadModelContainer(configuration:)` returns a model container, and a `ChatSession` wraps it so you can call `respond(to:)` repeatedly. The second call in that example, about a place to eat, relies on the session retaining conversation state, which is why the README prints two responses from one session rather than constructing a new one.
The freedom path is bringing your own downloader or tokenizer, or loading weights from local storage only. The README points to the using documentation for custom downloaders, alternative tokenizer packages and local-only weights. The trade-off is real: the macro path pulls in swift-huggingface and swift-transformers as dependencies, while the custom path keeps your dependency graph smaller at the cost of writing the conformance yourself.
Getting a session running: package and code
Installation is a normal Swift Package Manager dependency. The README gives `.package(url: "https://github.com/ml-explore/mlx-swift-lm", .upToNextMajor(from: "3.31.3"))` for the core package, then asks you to choose an integration package for downloaders and tokenizers.
The quick start example expands into three package dependencies: mlx-swift-lm, `https://github.com/huggingface/swift-huggingface` from 0.9.0, and `https://github.com/huggingface/swift-transformers` from 1.3.0. The target then lists products MLXLLM, MLXLMCommon and MLXHuggingFace from mlx-swift-lm, plus HuggingFace and Tokenizers from the two Hugging Face packages.
The code path is short. Import MLXLLM, MLXLMCommon, MLXHuggingFace, HuggingFace and Tokenizers, then call `#huggingFaceLoadModelContainer(configuration: LLMRegistry.gemma3_1B_qat_4bit)` and construct `ChatSession(model)`. The registry entry name tells you the model is a 4-bit quantized Gemma 3 1B. If you follow this example, the download happens through the Hugging Face integration, so the first run needs network access unless you have arranged local weights through the alternative integration path.
The 3.x breaking changes are the first thing to check
The README carries an important note: the `main` branch is a new major version, 3.x, and breaking changes were introduced in order to decouple from tokenizer and downloader packages. An upgrade guide is linked from the documentation site. If you have code written against the 2.x line, the integration surface you rely on is exactly the surface that moved.
The release list in the repository metadata shows 3.31.4 and 3.31.3 in 2026, alongside a 2.31.3 release in April 2026. That parallel numbering suggests the 2.x line received at least one release after 3.x was underway, but the supplied material does not state a support policy for 2.x, so treat continued 2.x maintenance as unverified.
There is a second, smaller operational detail worth noting: the README states that swift-format is used for consistent formatting and that CI pins it to 603.0.0. If you contribute patches, matching that version avoids format-check failures. This is a maintenance cost, not a runtime one.
Constrained output and the FoundationModels bridge
Two libraries in the repository go beyond plain text generation. MLXGuidedGeneration is described as grammar-constrained generation for any MLX model, supporting JSON Schema or EBNF. It is called out as a standalone primitive, which means it is not tied to the FoundationModels bridge or to a specific model family.
MLXFoundationModels is the bridge into Apple's FoundationModels framework. The pattern is to build an `MLXLanguageModel`, hand it to `LanguageModelSession`, and generate through the standard FoundationModels API. The README's example declares a `@Generable` struct, requests it with `session.respond(to:generating:)`, and notes that the response is grammar-constrained to that type's schema by combining the bridge with MLXGuidedGeneration. The listed capabilities are `.guidedGeneration`, `.vision`, `.toolCalling` and `.reasoning`.
The constraint here is version gating, and it is strict. The bridge requires the macOS, iOS or visionOS 27.0 SDK, and the README's own sample wraps the model construction in `if #available(iOS 27.0, macOS 27.0, visionOS 27.0, *)`. The `@Generable` struct in that sample is annotated for iOS 26.0 and later. If your deployment target is below 27.0, this library is not available to you and you should plan on the MLXLLM and MLXVLM path instead.
Where mlx-swift-lm is the wrong tool
The package is built on MLX Swift, which is an Apple array framework. That makes it a poor fit for anything that has to run on non-Apple hardware, and the supplied material offers no cross-platform story. A team serving many concurrent users from Linux hosts should look elsewhere; nothing here describes a batching server, request queue or HTTP layer.
The second limitation is dependency coupling. The convenient path depends on swift-huggingface and swift-transformers, and the 3.x release exists precisely because the project wanted to decouple from tokenizer and downloader packages. That decoupling is incomplete in practice: the README's quick start still names both Hugging Face packages as required dependencies. If your organisation restricts third-party dependencies, the custom conformance path is the escape hatch, and it is more work.
Third, the material does not state supported OS versions for the core MLXLLM and MLXVLM libraries, only for the FoundationModels bridge. If you need a specific minimum deployment target, that is something to confirm in the source documentation rather than assume.
How this differs from llama.cpp bindings
The obvious alternative for on-device inference in a Swift app is a binding to llama.cpp, which is a C/C++ inference engine with its own GGUF weight format and its own quantization schemes. The difference in approach is where the model execution lives. llama.cpp implements its own kernels and its own file format, so your weights come from the GGUF ecosystem. MLX Swift LM runs on MLX Swift, so weights come from the MLX and Hugging Face ecosystems, which is why the quick start pulls a registry entry rather than a file path to a GGUF.
That difference propagates into features. Grammar-constrained generation here is exposed as a standalone primitive over JSON Schema or EBNF, and the FoundationModels bridge lets an MLX model appear as a `LanguageModelSession` backend. A llama.cpp binding would not give you either of those without additional glue. On the other side, llama.cpp has a longer history of running outside Apple platforms, which matters if your inference code is shared with a backend service.
A second alternative is to use Apple's own FoundationModels framework directly with its built-in models. That gives you the system model with no download and no MLX dependency, but you give up choosing the model, and the README's bridge exists specifically because you might want an MLX model behind that same API.
Licence, maintenance and what to verify before adopting
The repository is MIT licensed, which is permissive and imposes no copyleft obligation on your application. This is not legal advice; confirm the licence text and any third-party dependency licences (swift-huggingface, swift-transformers, MLX Swift) against your own policy before shipping.
Maintenance cost centres on the model integration surface. The README points to a porting and implementing models guide, which implies that adding an architecture is expected work rather than an edge case. If you depend on a model family that is not already in the registry, budget for that effort. The 3.x breaking changes are the other recurring cost: the upgrade documentation exists because tokenizer and downloader decoupling forced code changes, and the same kind of change could recur at the next major version.
Before adopting, verify three things against the linked documentation. First, the exact migration steps in the upgrade guide for whichever tokenizer and downloader packages you use. Second, whether you need MLXGuidedGeneration or MLXFoundationModels at all, since the latter is gated on the 27.0 SDK and adds a hard floor to your deployment target. Third, how you will supply weights, because the macro path downloads from Hugging Face at runtime while the alternative path supports local-only weights.
Editorial conclusion
Adopt mlx-swift-lm if you are shipping a Swift app on Apple Silicon and want local LLM or VLM inference without writing your own Metal kernels; the MLXHuggingFace macros plus LLMRegistry give you a working ChatSession in a few lines. Do not adopt it if you need a language-agnostic serving layer or run on non-Apple hardware, because the package is built on MLX Swift and Apple frameworks. Before committing, verify the exact 3.x migration steps in the upgrade documentation for your tokenizer and downloader packages, and check which of MLXFoundationModels, MLXGuidedGeneration and MLXEmbedders you actually need, since the FoundationModels bridge requires the macOS/iOS/visionOS 27.0 SDK.
Community notes