LLM Foundry: Databricks' Training Stack for MPT and DBRX Models
LLM training code for Databricks foundation models
At a glance
- What is it?
- LLM Foundry is the Python codebase behind Databricks' MPT and DBRX models. It wraps Composer for training, finetuning, evaluation, and inference, but its tight coupling to MosaicML tooling shapes who should adopt it.
- Who is it for?
- Adopt LLM Foundry if you plan to train or finetune MPT or DBRX models and are comfortable with the MosaicML/Composer ecosystem. It is a poor fit if you need a framework-agnostic training pipeline or if you want to avoid the MosaicML platform tie-in.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 174 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What LLM Foundry Actually Is
LLM Foundry is a Python repository from Databricks that contains code for training, finetuning, evaluating, and deploying large language models. It is not a standalone training library. It builds on Composer, which is MosaicML's deep learning training library, and it is designed to work with the MosaicML platform. The repository is organized into source code under llmfoundry/ and scripts under scripts/ that cover data preparation, training, inference, and evaluation. The primary audience is engineers who want to reproduce or extend the training recipes used for Databricks' open models, specifically MPT and DBRX. If you are not working with those model families or Composer, much of this codebase will be irrelevant to you.
The Models: MPT and DBRX
The repository is the training code for two model families. MPT, or Mosaic Pretrained Transformers, are GPT-style models that use Flash Attention for efficiency, ALiBi for context length extrapolation, and stability improvements to reduce loss spikes. The README lists several MPT variants, from MPT-7B with a 2048 token context to MPT-30B with 8192 tokens, and a StoryWriter variant that reaches 65536 tokens. DBRX is a Mixture-of-Experts model with 132B total parameters and 36B active parameters. It was trained with optimized versions of Composer, LLM Foundry, and MegaBlocks. Both DBRX Base and DBRX Instruct have a 32768 token context. The DBRX weights are under the Databricks Open Source License, not Apache-2.0, and some MPT chat models are not licensed for commercial use. That licensing split matters: the repository itself is Apache-2.0, but the models you train with it may carry separate restrictions.
How the Training Pipeline Fits Together
The data flow starts with raw text that must be converted into StreamingDataset format. The scripts/data_prep/ directory contains conversion scripts for this purpose. StreamingDataset is a MosaicML format that streams data from object storage during training, rather than loading it all into memory. After conversion, the train/ scripts handle training or finetuning of HuggingFace and MPT models from 125M to 70B parameters. The training itself runs on Composer, which handles distributed training, optimization, and checkpointing. There are also benchmarking scripts under train/benchmarking to profile training throughput and MFU, or model flops utilization. For inference, scripts/inference/ converts models to HuggingFace or ONNX format and provides generation scripts like hf_generate.py and hf_chat.py. Evaluation is separate, under scripts/eval/, and runs academic or custom in-context-learning tasks. The mcli/ directory contains launch configurations for the MosaicML platform, meaning you can run these scripts locally or on MosaicML's managed infrastructure.
Getting It Running
The README points to the scripts/inference/README.md for interactive generation with HuggingFace models. To try a model locally, you would use hf_generate.py or hf_chat.py. The exact commands are not reproduced in the main README, but the pattern is to install llm-foundry from PyPI and run those scripts with a model name. For training, the TUTORIAL.md file is described as a deeper dive with example workflows. The repository structure suggests a typical flow: prepare data with data_prep scripts, write a YAML config for the model and training settings, then launch training with a Composer command. The MCLI tool, referenced as mcli/, is used to launch workloads on the MosaicML platform. If you want to run on your own cluster, you would need to install Composer and llm-foundry via pip. The PyPI version is current up to v0.22.0, released in July 2025. Without a more detailed README section, the exact installation command is not verifiable from the provided material, so you should check the TUTORIAL.md file in the repository.
Where LLM Foundry Falls Short
The most obvious limitation is the tight coupling to MosaicML infrastructure. The data format is StreamingDataset, which is not a universal standard. If your data pipeline produces Parquet or JSONL, you must convert it first. The training relies on Composer, which has its own API and configuration semantics. If your team already uses PyTorch Lightning or raw PyTorch with Hugging Face Trainer, adopting LLM Foundry means learning a new stack. The repository is also oriented toward large-scale training, with benchmarking scripts for throughput and MFU. That suggests it is designed for multi-GPU or multi-node setups. Running a small finetune on a single GPU might be possible, but the documentation emphasizes MPT and DBRX scale, and the MPT models start at 7B parameters, which is heavy for consumer hardware. Another failure mode: the README mentions Flash Attention as a feature, which requires compatible GPUs. On older hardware, training may not work at all.
Alternatives and How They Differ
The closest alternative is Hugging Face Transformers with its Trainer API. Transformers supports a wide range of architectures beyond MPT and DBRX, including LLaMA, Mistral, and others. Its approach is different: it has a larger ecosystem, more community examples, and a neutral data format (datasets or plain text). Transformers does not require a specific streaming format or a proprietary platform. However, it lacks the built-in benchmarking for MFU and the optimized training recipes that LLM Foundry provides for MPT and DBRX. Another alternative is NVIDIA NeMo, which is a framework for large-scale training and has its own data pipeline and configuration system. NeMo is more opinionated about model architectures and uses Megatron-style parallelism. LLM Foundry is more focused on the MosaicML stack, while NeMo is tied to NVIDIA's ecosystem. The choice depends on whether you want to stay within the Databricks/MosaicML world or prefer a broader framework.
Maintenance and Upgrade Considerations
The repository is actively maintained, with the last push in March 2026 and recent releases v0.22.0 in July 2025, v0.21.0 in May 2025, and v0.20.0 in April 2025. That cadence suggests regular updates, but it also means you must track version compatibility with Composer and PyTorch. The README does not list explicit version requirements, so you will need to check the release notes or setup.py. The license is Apache-2.0, which permits commercial use and modification, but the model weights for DBRX and some MPT variants have separate licenses. If you train a model with LLM Foundry, the resulting weights may be subject to the model's license. For example, MPT-7B-Chat is not licensed for commercial use. That is a legal consideration, not a technical one, but it affects deployment. The upgrade cost is moderate: because the codebase is tied to Composer and MosaicML, you cannot easily swap out components. Each new release may change config schemas or script interfaces, so you should test upgrades in a staging environment.
Editorial conclusion
Adopt LLM Foundry if you plan to train or finetune MPT or DBRX models and are comfortable with the MosaicML/Composer ecosystem. It is a poor fit if you need a framework-agnostic training pipeline or if you want to avoid the MosaicML platform tie-in. Before committing, verify that your dataset can be converted to StreamingDataset format, that your hardware supports Flash Attention, and that the latest release v0.22.0 matches your PyTorch and Composer versions. Check the TUTORIAL.md file for workflow examples and the mcli/ directory for platform launch configs. If you need a more general training framework, consider alternatives like Hugging Face Transformers with Trainer, which has a broader community and no MosaicML dependency.
Community notes