Tinytorch

The main file stays under 1000 lines, with a promise to revert if it grows. Tinytorch positions itself between micrograd and tinygrad on a deliberately playful power scale.

Tinytorch calls itself a really tiny autograd engine, and the README jokes that it is the newest ML framework users probably do not need. It is backed by numpy, and the main file is kept under 1000 lines with a stated intention to revert changes if the count passes that. That ceiling is the design constraint, and it is what keeps the code readable as an educational project. Readability over reach is the trade being made on purpose.

There is a practical ceiling to numpy. The README warns that numpy is too slow to train an LLM, and points anyone who wants to scale up at jax as a faster numpy drop in. For small experiments the plain engine is fine, and the jax note only matters once you outgrow them.

The project positions itself on a power level scale. Karpathy's micrograd sits at 1.0 and is described as really simple, Tinytorch at 3.14 is simple but capable of a lot, and tinygrad at 69 is no longer simple. A linked development blog documents building tensors, the backward pass, refactoring and cleanup, and broadcasting, so the construction is traced step by step rather than presented as a finished artifact. The blog parts follow the building of the engine in order, which makes the project genuinely useful for learning autograd internals. It reads like software made to be read, not just shipped.

• Source 1 · Project repository
• Source 2 · README