Olmo-core is the PyTorch training stack behind the OLMo model family

A training stack, not an inference library

Olmo-core is described as building blocks for OLMo modeling and training, and that phrase is the accurate one. The package covers the model definitions, the data pipeline, the optimizers and the training loop for the OLMo family, and it also carries a beta path for autoregressive generation inside the project itself.

The users are research engineers and platform teams who run pretraining themselves. The evidence for that audience is in the artefacts rather than in prose: the repository ships official training scripts for released model sizes, and the project documentation is hosted on readthedocs with an API reference, an examples badge pointing at src/examples, and a link to an arXiv paper describing the work.

The name on PyPI is ai2-olmo-core, and the project describes itself as the core training module for the Open Language Model. A reader who only wants to run a released checkpoint has better options in the same ecosystem, since the project documents inference through Hugging Face transformers and through vLLM without requiring Olmo-core in the loop at all.

So the boundary is clean. Olmo-core is for the part of the work between a dataset and a set of checkpoint weights, and it stays useful there even if you never touch its generation code.

Three ways in, and they are not equivalent

Installation has a fixed first step. The documentation asks you to install PyTorch first, following the instructions for your operating system and hardware, because the CUDA and ROCm builds of torch are not interchangeable and the project does not bundle them.

For development, the documented path is a clone and an editable install with all extras.

git clone https://github.com/allenai/Olmo-core.git
cd Olmo-core
pip install -e .[all]

The [all] extra pulls in the optional dependency groups, which the pyproject file defines per feature area. Among the named groups are beaker for the Beaker launch path, comet for comet_ml, dion for a Linux only dependency, eval for ai2-olmo-eval, and fa4 for a particular attention backend. Each group is optional, so a minimal install is possible if you never touch those paths.

If you are consuming the package rather than hacking on it, the PyPI route is a single command.

pip install ai2-olmo-core

The package requires Python 3.10 or newer and pins a narrow torch range, torch>=2.10.0,<2.14. That upper bound is the detail worth noting before you install, because a cluster already running a different torch build will need either a different environment or a rebuild of torch to match.

Optional kernels decide which features you can switch on

A long list of optional dependencies in the installation section is really a list of hardware features. The attention backends are flash-attn, ring-flash-attn and TransformerEngine. Liger-Kernel provides a low-memory fused-linear loss implementation. torchao is needed for float8 training. grouped_gemm is required for dropless mixture-of-experts models, and the documentation warns that it may need to be compiled from source until PR #21 is released, post v0.1.6. QuACK is listed for some CuTe-based kernels.

The practical consequence is that the model code and the accelerator features are decoupled. Nothing in the core package forces a particular attention implementation, but each kernel that you want to use has to be installed separately, and the grouped_gemm case shows what that costs: an extra build step from source when a tagged release is not yet out.

The dependency list in pyproject tells the same story from the other direction. The required set is numpy, torch, cached-path, filelock, requests, packaging, dataclass-extensions, pyyaml, rich, safetensors, importlib_resources, bettermap and pandas. Read on its own that is a normal research Python stack, with cached-path and safetensors handling data and checkpoint IO and rich handling terminal output. It is a package that expects to be installed into a working GPU environment, not a self-contained binary.

Official training scripts run under torchrun

The scripts used to train released models live in src/scripts/official, and they are meant to be launched with torchrun, or with the project's Beaker launch CLI if you have Beaker access.

A training launch looks like this.

torchrun --nproc-per-node=8 src/scripts/official/OLMo2/OLMo-2-0325-32B-train.py \
  --save-folder=/path/to/save/checkpoints

The eight in --nproc-per-node is the number of processes on one node, and the script path selects the model and size. Configuration is overridden from the command line with dotted keys, so a different learning rate is passed by naming the config path rather than editing the file.

torchrun --nproc-per-node=8 src/scripts/official/OLMo2/OLMo-2-0325-32B-train.py \
  --save-folder=/path/to/save/checkpoints \
  --train_module.optim.lr=6e-3

Annealing from an existing checkpoint is a separate script, and it takes the checkpoint path as an argument.

torchrun --nproc-per-node=8 src/scripts/official/OLMo2/OLMo-2-0325-32B-anneal.py \
  --save-folder=/path/to/save/checkpoints \
  --checkpoint=https://storage.googleapis.com/ai2-llm/peteish32/step721901

Two model families are covered. The OLMo-2 directory holds the training script and model card for the 32B models, and the OLMo-3 directory covers the 7B and 32B models. The scripts are Python files named for the model, which makes it easy to check what you are launching before you burn cluster hours on it.

The published Docker images deliberately omit the package

The project publishes Docker images and, unusually, warns against expecting a ready to run installation from them. Two limitations are stated directly. The images do not come with the Olmo-core package installed, only its dependencies, and the stated reason is to accommodate regular code changes. And they may not work on your own cluster if you have different hardware or driver and CUDA versions.

The images are described as containing all core and optional dependencies, and they are regularly tested on the in-house H100 clusters. That sentence is a statement about where they were validated, not a guarantee for your hardware.

If the published images do not fit, the documented fallback is to adapt the project's own Dockerfile, which lives at src/Dockerfile, and build your own. The Makefile shows what that environment is assembled from: CUDA version 12.9.1 with a matching nvcc version of 12.9.86, Python 3.12, and a torch version of 2.13.0. The wheel install channel is used rather than a source build.

That last combination is worth checking against your cluster. The images pin a specific CUDA and torch pair, and the Makefile even carries a note about keeping the versions in pyproject and the CI setup action in step when torch is upgraded.

Three inference paths, and only one of them needs the package

For inference the project documents three routes, and the difference in approach between them is the point.

The first is the Hugging Face transformers integration, which loads a checkpoint through the standard auto classes.

pip install transformers>=4.57.0

from transformers import AutoModelForCausalLM, AutoTokenizer
olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-1125-32B")
tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3-1125-32B")
message = ["Language modeling is "]
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
response = olmo.generate(**inputs, max_new_tokens=100, do_sample=True, temperature=1.0, top_p=0.7)
print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])

The second is vLLM, described in the project documentation as providing high-throughput inference for OLMo models, used here for offline batched inference.

pip install vllm>=0.11.0

from vllm import LLM, SamplingParams
llm = LLM(model="allenai/Olmo-3-1125-32B")
sampling_params = SamplingParams(temperature=1.0, top_p=0.7)
prompts = ["Language modeling is"]
outputs = llm.generate(prompts, sampling_params)

The third path is the one inside Olmo-core, where autoregressive generation is supported directly and the project ships a chat-loop demo for interactive use. The documentation labels that capability beta, and that word is the only warning attached to it.

The choice is between a familiar integration, a throughput oriented engine that takes over batching and scheduling, and the in-repo path that keeps everything on one code base. Only the third needs Olmo-core installed.

A 3.0 release, pinned dependencies and Apache-2.0

The repository is not archived, and the last push was on 2026-10-18, so the code base is moving. Releases came at v3.0.0 on 2026-10-01, v2.6.0 on 2026-08-11 and v2.5.0 on 2026-04-03, a move from the 2.x line to a major version inside a single year.

The upgrade cost of that jump is shaped by the dependency pins rather than by the API alone. torch is bounded above, Python has a floor of 3.10, and the evaluation dependency is pinned to an exact version, ai2-olmo-eval==0.9.0. Style and lint tooling are pinned tightly as well, including ruff at 0.15.9 and furo at 2024.8.6, which tells you the project treats its own tooling versions as fixed.

The Makefile shows the checks a contribution has to pass: isort and black for style, ruff for lint, and mypy over src/ for types. The docs target rebuilds Sphinx with sphinx-autobuild, and the build target runs python -m build followed by a metadata check script.

The licence is Apache-2.0, which permits use, modification and redistribution in products including closed source ones, and the repository carries a separate THIRD_PARTY_NOTICES.md file for the licences of bundled dependencies. Optional kernels such as flash-attn, torchao and Liger-Kernel bring their own terms, and that notices file is the place to check them.

Editorial conclusion

Olmo-core is worth adopting for a group that pretrains OLMo style models itself and wants the training and inference code in one place, and it is the wrong dependency for a team that only loads an existing checkpoint. Verify three things first: that the ai2-olmo-core version on PyPI matches the torch range your cluster has, since the package pins torch>=2.10.0,<2.14, that the optional kernels you need are installed for your GPU, and that the Docker image you pull actually contains the package, because the published images ship dependencies only.

Frequently asked questions

Official sources

1. allenai/Olmo-core on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases