tinygrad: A Hackable Deep Learning Stack Between PyTorch and Micrograd

What tinygrad Solves and Who It Is For

The project is maintained by tiny corp, and the repository is active, with releases like v0.14.0 in August 2026. It is licensed under MIT, which is permissive for both academic and commercial use. The README explicitly welcomes contributors but sets strict rules: no code golf, no docs changes from newcomers, and any speedup claim must be benchmarked. This tells you the project values simplicity and readability above all.

The Architecture: Laziness, IR, and JIT

tinygrad's core mechanism is laziness. Operations on tensors are not executed immediately; they are recorded into a graph that gets fused and lowered into kernels. The README gives a concrete example: a matmul written as a chain of reshape, multiply, and sum, and with DEBUG=3, it fuses into one kernel. With DEBUG=4, you see the generated code. This lazy evaluation is what enables the compiler to optimize the computation without user intervention. The stack includes an IR and compiler that fuse and lower kernels, plus a JIT called TinyJit that captures and replays kernels. There is also a device graph for batched execution. This architecture is inspired by JAX's IR-based autodiff and TVM's scheduling, but tinygrad ships the front-end framework as well, unlike TVM which is only a compiler. The visible IR is a key differentiator: you can inspect the graph and the generated code, which is impossible in PyTorch without diving into C++.

Accelerator Support: The 25-Op Rule

tinygrad supports a wide range of accelerators: OpenCL, CPU, METAL, CUDA, AMD, NV, QCOM, and WEBGPU. The README claims that adding a new accelerator is easy because it only needs to support about 25 low-level ops. This is a deliberate design choice: keep the primitive set small so that porting is tractable. To check which accelerator is default on your system, run: python3 -c "from tinygrad import Device; print(Device.DEFAULT)". This command is useful before you start, because not all accelerators are equal in maturity. The README lists them as supported, but it does not say which ones are production-ready. For example, CUDA and METAL are likely well-tested, but QCOM or WEBGPU may be experimental. This is a limitation to verify before relying on a specific backend for real work.

Getting Started: Installation and First Steps

The recommended installation is from source. You clone the repository and run: git clone https://github.com/tinygrad/tinygrad.git, then cd tinygrad, then python3 -m pip install -e. This gives you the latest master. There is also a direct pip install from git: python3 -m pip install git+https://github.com/tinygrad/tinygrad.git. The README warns that tinygrad is not 1.0 yet, so expect API changes. The quick example shows a familiar PyTorch-like pattern: create tensors, do a matmul, call backward, and print gradients. The code is nearly identical to PyTorch, which lowers the learning curve. For a full training example, the README points to examples/beautiful_mnist.py, which claims to get 98% accuracy in about 5 seconds, though I cannot verify that claim from the README. The training loop uses nn.optim.Adam, a Context for training mode, and sparse_categorical_crossentropy. This is enough to write a real model.

Limitations and Failure Modes

The most obvious limitation is the lack of full functional transforms. The README compares tinygrad to JAX and admits that it has fewer functional transforms, with no full vmap or pmap yet. If your work depends on vectorized map over batch dimensions, tinygrad will not cover it. Another limitation is the maturity of the code outside the core tinygrad/ folder. The contribution guidelines state that code outside core is not well tested, so you should not change it unless it is broken. This means that the nn, optim, and datasets modules may be less stable than the tensor core. The project is also pre-1.0, so APIs can shift between releases. A concrete failure mode: if you try to use a feature that relies on a less common accelerator, you may hit a bug that is not fixed because the maintainers prioritize the core. Finally, the README's contribution rules are hostile to AI-generated code, which is fine for human contributors but means the project will not accept automated patches, so you must write your own fixes.

Comparison with PyTorch, JAX, and TVM

The README gives a direct comparison. PyTorch is similar in ergonomics: eager tensor API, autograd, optim, and basic datasets. The difference is that tinygrad's compiler and IR are visible and hackable, while PyTorch's are hidden. JAX is similar in IR-based autodiff and function-level JIT, but tinygrad has fewer functional transforms. TVM does scheduling and codegen, but tinygrad also provides the front-end framework. The real difference in approach is that tinygrad is a single, coherent Python codebase where you can trace a tensor operation from the user call down to the generated kernel. PyTorch separates the Python front-end from a C++ backend. JAX relies on XLA, which is a massive external dependency. TVM is a compiler project, not a framework. If you want to modify the compiler itself, tinygrad is the only one of these where you can do it in a weekend. That is the core value proposition.

Maintenance and Upgrade Costs

Maintenance cost is tied to the project's volatility. Since tinygrad is pre-1.0, each release may break APIs. The releases are frequent: v0.14.0 in August 2026, v0.13.0 in May 2026, v0.12.0 in January 2026. That is roughly three releases in eight months, which suggests active development but also churn. You will need to read the release notes and update your code accordingly. The codebase is intentionally small, which reduces maintenance burden compared to PyTorch, but the lack of stable interfaces means you cannot treat it as a drop-in dependency. The license is MIT, so you can fork and modify freely, but the contribution rules make it hard to upstream changes unless they are small and clearly beneficial. If you adopt tinygrad, plan to budget time for keeping up with changes, or pin a specific commit and maintain your own fork. The documentation lives in the docs/ directory, and the README says the docs website is built from there, but it is not clear how current the docs are relative to the latest release.

Editorial conclusion

Adopt tinygrad if you are a researcher, compiler engineer, or student who wants to read and modify the entire stack from tensor ops to codegen, and you value simplicity over raw performance or feature completeness. Skip it if you need production-grade training loops, extensive functional transforms like vmap, or a stable API. Before adopting, verify that your accelerator is supported (check Device.DEFAULT), review the current state of the core tinygrad/ folder, and confirm that the documentation covers your use case, since the project is pre-1.0 and the code outside core is not well tested.

Official sources

1. Official README
2. Project repository
3. Release notes