mlx-lm-lora: Preference Optimization and QAT on Apple Silicon
Train Large Language Models on MLX.
At a glance
- What is it?
- The project wraps MLX-LM with a training CLI that covers LoRA, DoRA, full fine-tuning, and a long list of preference objectives, plus quantization aware training for SFT, DPO, and ORPO. The breadth is real, but the documentation leaves the practical details of dataset formats and memory tuning to a separate examples repository.
- Who is it for?
- Adopt mlx-lm-lora if you already run MLX on an Apple Silicon machine and your objective is one of the supported set, particularly if you want QAT for SFT, DPO, or ORPO, since that combination is uncommon. Do not adopt it if you need multi-GPU training, a stable API across releases, or documentation that explains dataset schemas without reading example notebooks in another repository.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 10 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What mlx-lm-lora is for
The README states the goal directly: train large language models locally on Apple Silicon using MLX. That places the project in a narrow slot. It is not a general fine-tuning framework that happens to have an Apple backend. It targets people who own a Mac with unified memory and want to adapt a model without renting GPU time elsewhere. The README lists Llama, Mistral, Qwen, Gemma, OLMo, OLMoE, MiniCPM and MiniCPM3 as model families that work, with the qualifier that training works with all models supported by MLX-LM. That qualifier matters more than the list. The real constraint is upstream: whatever MLX-LM can load, this project can train. The audience is therefore someone already inside the MLX ecosystem, typically running a single machine, often experimenting with small instruction-tuned checkpoints such as the Qwen2.5-0.5B example used in the quick start. The project also assumes familiarity with preference optimization terminology. It ships SFT, DPO, CPO, ORPO, GRPO, GSPO, Dr. GRPO, DAPO, Online DPO, XPO, RLHF Reinforce with KL, and PPO as named training algorithms. Someone who has only done supervised fine-tuning will find the option list long and the guidance about which one to pick absent from the README.
The training CLI and how a run is assembled
The entry point is a single console command, mlx_lm_lora.train. Running it with --help prints the options. A minimal invocation passes a model, the --train flag, a dataset, and an iteration count:
mlx_lm_lora.train --model <model> --train --data mlx-community/wikisql --iters 600
Everything else is optional or has defaults. The README also documents a config path, taken with -c or --config, pointing at a YAML file. The precedence rule is stated explicitly: command-line flags override corresponding values in the config file. That is a sensible arrangement for experimentation, since you can keep a base YAML and vary a single flag per run. The architecture implied by the material is a thin orchestration layer over MLX-LM: the model loading, tokenizer, and generation come from upstream, while this project supplies the training loop, the loss functions for each algorithm, and the adapters. The --train-type flag selects between lora, dora, and full, with lora as the default. Quantized training is described as QLoRA with 4-bit, 6-bit, or 8-bit quantization. SFT adds a further knob, --sft-loss-type, shown in the README with the value dft, and --mask-prompt, which restricts the loss to assistant responses. Those two flags are the ones most likely to change results on instruction data, because masking prompt tokens stops the model from learning to reproduce user turns.
Quantization aware training: the part worth a closer look
The most distinctive feature described is QAT. The README explains the mechanism in one sentence: QAT projects trainable weights onto a quantized grid after each optimizer update, simulating quantization effects during training. The stated benefit is improved quantized model performance. Support is limited to SFT, DPO, and ORPO. The flags are explicit. --qat-enable turns projection on. --qat-bits sets the width, default 8. --qat-group-size defaults to 64, and the README notes that 0 means per-tensor. --qat-mode defaults to affine. --qat-start-step defaults to 1, and --qat-interval defaults to 1, meaning projection happens after every optimizer step unless you change it. The README gives three worked examples, one per supported algorithm. The SFT example uses --qat-bits 4 with group size 64 and the default start step and interval. The DPO example sets only --qat-bits 4. The ORPO example uses --qat-bits 8 with --qat-group-size 32. The design choice to expose --qat-start-step and --qat-interval is interesting and not explained further. Projecting from step 1 onward is the default, which means the model trains against quantized weights from the beginning rather than annealing into them. Whether that is better than a later start is not addressed in the material, and the README does not report any comparison. The bit range is stated as 4 to 16 bits in the feature summary, while the flag description says the default is 8.
The preference algorithm list and its cost
Thirteen training algorithms are listed, from SFT through PPO. Several are recent variants with overlapping names: GRPO, GSPO, Dr. GRPO, DAPO, and XPO all sit in the group-relative or preference-optimization space. The README names them and gives the table of contents entries for each, but the supplied material does not include the per-algorithm sections beyond headings. That is the main documentation gap. A reader can learn that GRPO exists here, and that it is described as group-based reinforcement training in the examples repository listing, but cannot learn from the README what data each algorithm expects. One exception is FTPO, also called Antidoom, described as final-token preference optimization for repairing repetition loops. That is a specific, narrow purpose, and it is the clearest statement of intent attached to any algorithm in the material. There is also a feature for training a custom preference model, intended for online preference training. Again, the mechanism is not described in the README beyond the one-line summary. The practical consequence is that the algorithm list functions as a menu without descriptions. Choosing among DPO, CPO, ORPO, and XPO requires outside knowledge or reading the separate notebooks repository, which is where the README says every notebook, YAML config, and walkthrough now lives.
Installation, configuration and dataset uncertainty
Installation is a single pip command: pip install -U mlx-lm-lora. There is no mention of a minimum Python version, an MLX version floor, or macOS requirements in the supplied material. That is a gap worth noting before you start, because MLX itself has platform constraints and the project inherits them. The README has a Configuration section in its table of contents, but the supplied text does not include its contents, so the only configuration facts available are the -c/--config flag, the YAML format, and the override precedence. Dataset formats are similarly listed as a section without the body being present. The quick start uses mlx-community/wikisql, and the SFT example uses mlx-community/hermes-3, both as --data values. That suggests datasets can be referenced by Hugging Face identifier, but the README does not state whether local paths are also accepted, what column names are expected, or how preference pairs must be structured for DPO and its relatives. Anyone planning a real run should treat the dataset format question as unresolved until they read the examples repository. The README's own pointer is emphatic about where that material lives, which reads as a deliberate split rather than an oversight.
Limitations and the wrong-tool case
The first limitation is hardware. Training runs on Apple Silicon through MLX. There is no CUDA path described, no distributed training, and no multi-node story in the material. If your model does not fit in unified memory, this project is the wrong tool regardless of how many algorithms it offers. The second limitation is the documentation split. Core reference material for dataset formats, memory optimization, evaluation and generation, and performance comparison exists as section headings in the README, but the bodies are not in the supplied text, and the notebooks have moved to a separate repository. A team that needs a self-contained reference will be following links. The third is API stability. The release history shows v2.1.0 in April, v3.0.0 in July, and v3.1.2 in August, all within the same year. A major version bump between minor releases signals that flags and defaults can move. Pinning a version is reasonable here. The fourth is scope creep in the algorithm list. Supporting thirteen objectives across four training types multiplies the test surface, and the README does not describe a test suite. The fifth is that QAT applies only to SFT, DPO, and ORPO. If your objective is GRPO or PPO, the QAT flags are not available to you according to the stated support list.
How this differs from the MLX-LM baseline and from PEFT-style tooling
The obvious alternative is MLX-LM itself, the upstream library this project builds on. MLX-LM provides the model loading and a LoRA fine-tuning path. mlx-lm-lora adds the preference and reinforcement algorithms, DoRA and full fine-tuning modes, quantized training, and QAT projection. The difference in approach is that mlx-lm-lora treats training as a configurable pipeline with a single entry command and a YAML layer, whereas the upstream tool is closer to a set of scripts. If you only need supervised LoRA on a small model, the upstream path is fewer moving parts and one fewer dependency to track. The second alternative is PEFT with TRL on PyTorch. That stack has broader hardware support, including CUDA and multi-GPU, and a larger body of documentation. The difference is not just performance but platform: PEFT does not run on MLX, so on a Mac you are choosing between a Metal-native path and a PyTorch path that may fall back to CPU for some operations. The third alternative is not training at all. For many adaptation tasks, retrieval or prompt construction avoids the failure modes of fine-tuning entirely. That is worth stating because the README's framing assumes training is the goal, not a means.
Maintenance, licence and what to verify first
The repository is Apache-2.0, which permits commercial use, modification, and redistribution provided the licence and notices are preserved. It also includes an explicit patent grant. That is a permissive arrangement, and it is compatible with the MLX-LM dependency it builds on, though you should confirm the licence of any model weights you train separately, since model licences vary and are not covered by this project's licence. This is not legal advice. On maintenance, the commit history shows a push in September 2026 and three releases across the year, so the project is active. The cost of upgrading is the usual one for a fast-moving training tool: flag names, defaults, and supported algorithm combinations can change between major versions, and the v2 to v3 jump means an upgrade may require editing YAML configs and command lines. Pinning the version in your environment and re-reading the QAT flag defaults after any bump is the concrete step. The things to verify before adopting are narrow and checkable: that your target model appears in the MLX-LM support list for your installed version, that your dataset matches the schema your chosen --train-mode expects, and that a short run at your intended --batch-size completes without exhausting memory. If all three hold, the QAT path for SFT, DPO, or ORPO is the feature that distinguishes this project from the upstream MLX-LM scripts.
Editorial conclusion
Adopt mlx-lm-lora if you already run MLX on an Apple Silicon machine and your objective is one of the supported set, particularly if you want QAT for SFT, DPO, or ORPO, since that combination is uncommon. Do not adopt it if you need multi-GPU training, a stable API across releases, or documentation that explains dataset schemas without reading example notebooks in another repository. Before committing, verify that the model you intend to train is supported by the installed MLX-LM version, that your dataset matches the format the chosen train-mode expects, and that a short run completes at the batch size you plan to use.
Community notes