What is tairov/llama2.mojo?

A one-file Llama 2 inference implementation written in Mojo, aimed at people who want to read the whole transformer in one sitting and run it on CPU. The trade-off is narrow model support and a toolchain that is still moving.

Can I use tairov/llama2.mojo commercially?

Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.

Is tairov/llama2.mojo still maintained?

Yes. The repository last received commits 3 days ago.

What programming language is tairov/llama2.mojo written in?

Mainly Mojo, according to GitHub's language statistics.

llama2.mojo: Llama 2 inference in one Mojo file

At a glance

What is it?: A one-file Llama 2 inference implementation written in Mojo, aimed at people who want to read the whole transformer in one sitting and run it on CPU. The trade-off is narrow model support and a toolchain that is still moving.
Who is it for?: Adopt llama2.mojo if you want to read or modify a complete Llama 2 inference path in one file and you are comfortable installing the Mojo 1.0.0 toolchain first. Skip it if you need a broad model zoo, GPU serving, or a stable API, since the README lists only the stories checkpoints and TinyLlama-1.1B-Chat-v0.2 as supported.
Can I use it commercially?: Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?: Yes. The repository last received commits 3 days ago.
What is it written in?: Mainly Mojo, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.

DEEP OPEN-SOURCE ANALYSIS

Hysen Labs Editorial · September 15, 2026 · tairov/llama2.mojo

The problem llama2.mojo solves, and who it is for

Most Llama 2 inference code is spread across a project: a tokenizer module, a model loader, a matmul kernel, a sampling loop, and a build system that ties them together. Reading it end to end takes an afternoon. llama2.mojo collapses that into a single file, llama2.mojo, at the repository root. The README frames the project as a port of the author's earlier llama2.py, moved to Mojo so the same logic can use SIMD and vectorization primitives. The stated goal is speed on CPU, not feature coverage.

The audience is therefore narrow. It suits someone who wants to see how a Llama 2 forward pass is actually written, or who wants to change one part of it (the sampler, the matmul, the worker pool) without hunting through a package tree. It also suits anyone curious about Mojo itself, since the whole program is a working example of the language rather than a toy snippet. It is not aimed at teams serving models in production, and the README does not present it that way.

What the single-file inference path actually contains

The repository root holds llama2.mojo alongside tokenizer.bin, t260.bin, and a tests/ directory. The README's example output shows what the program prints at startup: a line reporting the number of parallel workers and the SIMD width, then a line reporting total bytes read and the estimated checkpoint size, then the model shape as n layers and vocab size. That output is the clearest available description of the internal structure, since the README does not document the code layout in prose.

The parallelism is explicit. The -j flag sets the number of parallel workers, defaulting to the number of performance cores, and the README describes those workers as persistent threads that spin between tokens. That is a design choice with a cost: idle spinning burns CPU between tokens instead of parking the threads. The README warns directly not to ask for more workers than you have cores.

Performance claims in the README are the author's own benchmark tables, run on an Apple M1 Max, an Ubuntu 26.04 VM with 4 vCPU, and an i7-8700, all with Mojo 1.0.0. The README states the Mojo version outperforms llama2.c by 30% in multi-threaded inference and llama.cpp on baby-llama CPU inference by 20%. Those numbers come from the project's own tables, not from an independent run.

Installing the Mojo toolchain and running your first prompt

The README says the quickest way to get the Mojo 1.0 toolchain is the modular Python package, which ships the mojo compiler together with MAX. The commands below create a virtual environment, install that package, and put the mojo binary on your PATH. After this, mojo --version should report Mojo 1.0.0.

bash

python3 -m venv ~/.modular-venv
~/.modular-venv/bin/pip install modular
export PATH="$HOME/.modular-venv/bin:$PATH"
mojo --version

Linking a Mojo executable needs a C compiler on the host. On Debian or Ubuntu the README gives apt-get install gcc for that. Then clone the repository and enter it.

bash

git clone https://github.com/tairov/llama2.mojo.git
cd llama2.mojo

The README's walkthrough downloads the smallest stories checkpoint from Hugging Face. The file is stories15M.bin.

bash

wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin

Finally, run the model with a prompt. The README's example uses a seed of 100, 256 steps, temperature 0.5, and the opening phrase of a story.

bash

mojo llama2.mojo stories15M.bin -s 100 -n 256 -t 0.5 -i "Once upon a time"

What you should see is the startup lines described earlier (parallel workers, SIMD width, checkpoint size, layer and vocab counts) followed by generated text. The README's sample output continues the prompt with a short story about a girl named Lily. The flags are -s for seed, -n for steps, -t for temperature, -i for the input prompt, -z for the tokenizer path (default tokenizer.bin), -j for worker count, and -pc to print config. Setting -n to 0 means max_seq_len.

Where llama2.mojo is the wrong tool

Model support is the first limit. The README lists exactly five checkpoints that were successfully executed: stories 260K, 15M, 42M, 110M, and Tinyllama-1.1B-Chat-v0.2. A Llama 2 7B or 13B checkpoint is not on that list, so anyone expecting to run a full-size Llama 2 with this file is outside what the project documents. The name suggests broader coverage than the supported models table delivers.

The worker model is the second limit. The -j flag defaults to the number of performance cores, and the README warns that workers spin between tokens. On a shared machine or a laptop on battery, that spinning is wasted work. Setting -j 1 avoids it, but the README's own Ubuntu benchmark table shows the cost: stories15M.bin drops from 327 tok/s with -j 4 to 154 tok/s with -j 1 on the same 4 vCPU VM. If you cannot give the process dedicated cores, you are choosing between wasted cycles and lower throughput.

Toolchain coupling is the third. The README pins the supported version to Mojo 1.0.0 and Modular 26.5.0. Mojo is a young language, and the repository carries an upgradelog.md at the root, which suggests the code has needed updating as the toolchain moved. A project that tracks a pre-1.0-era compiler will need attention when that compiler changes. There is also no release history retrieved for this repository, so there are no tagged versions to pin against.

How it differs from llama2.c and llama.cpp

The closest comparison is llama2.c, which the README benchmarks against directly. Both are single-file CPU inference implementations of the same model family, and both are meant to be read. The difference is the language and the optimization route. llama2.c is C and uses OpenMP for parallelism in the benchmarked configuration. llama2.mojo is Mojo and uses the parallelize facility that ships with MAX, plus SIMD and vectorization primitives, which is the mechanism the README credits for the performance difference.

The second comparison is llama.cpp, which the README benchmarks against on baby-llama CPU inference. llama.cpp is a much larger project with quantization support and a broad model zoo; llama2.mojo is one file with a fixed set of supported checkpoints. If you need quantized weights or a model that is not in the README's table, llama.cpp is the different approach: breadth and format support rather than a readable single file. The Ubuntu benchmark table in the README also shows llama2.mojo ahead of llama2.c only modestly on smaller models (327 versus 292 tok/s for stories15M.bin at 4 threads), so the gap is real but not uniform across hardware.

Licence, maintenance, and what an upgrade costs

The repository is MIT licensed, which is permissive and imposes few conditions on reuse. Note that the Dockerfile in the repository is a separate work: its header carries the Apache License v2.0 with LLVM Exceptions and a Modular copyright notice, and it is adapted from Modular's own example Dockerfile. If you copy that file rather than the Mojo source, you are dealing with a different licence. That is a factual difference in the files, not legal advice; check with your own counsel if the distinction matters to you.

The repository is not archived, and the last push was on 2026-09-12, three days before this writing. The README's benchmark table is dated the same day and notes a persistent worker pool build from PR #102, so the code and the numbers were updated together. There are no retrieved releases, which means upgrades happen by pulling the branch rather than by moving between tags. The practical cost of an upgrade is therefore re-running your build against a new Mojo version and re-checking the benchmark tables, since the README ties its numbers to Mojo 1.0.0 and Modular 26.5.0 specifically.

Editorial conclusion

Adopt llama2.mojo if you want to read or modify a complete Llama 2 inference path in one file and you are comfortable installing the Mojo 1.0.0 toolchain first. Skip it if you need a broad model zoo, GPU serving, or a stable API, since the README lists only the stories checkpoints and TinyLlama-1.1B-Chat-v0.2 as supported. Before committing, confirm that your machine can build a Mojo executable, which the README says requires a C compiler on the host, and check that the checkpoint you want appears in the supported models table.

Frequently asked questions

Which models can llama2.mojo run?

The README lists stories 260K, 15M, 42M, and 110M, plus Tinyllama-1.1B-Chat-v0.2, as the models that were successfully executed. Full-size Llama 2 checkpoints are not in that table.

How do I install llama2.mojo?

The README says to install the modular Python package, which ships the mojo compiler and MAX, then clone the repository and run mojo llama2.mojo with a downloaded checkpoint such as stories15M.bin. Linking a Mojo executable also needs a C compiler on the host.

What does the -j flag do in llama2.mojo?

It sets the number of parallel workers, defaulting to the number of performance cores. The README notes that workers are persistent threads that spin between tokens, so it warns against requesting more workers than you have cores.

Is llama2.mojo faster than llama2.c?

The README's own benchmark tables report llama2.mojo ahead of llama2.c in multi-threaded inference, for example 1283 versus 730 tok/s for stories15M.bin on an M1 Max. On a 4 vCPU Xeon VM the same model is closer, 327 versus 292 tok/s.

Official sources

Community notes