modern-llm-notebook

What the course covers

Modern LLM Notebook is a from scratch, notebook first course for understanding modern LLM systems. The idea is to build the core components yourself, from the tokenizer and transformer up through training, inference, alignment, and production. The README says the Chinese edition is the source edition and the English mirror is updated alongside it, with both English and Chinese documentation linked. The course is aimed at people who learn by running code rather than by reading theory alone. It is bilingual, with a course map that connects foundations, training, inference, frontier topics, and production systems. The August 2026 update rebuilt Part 3, Inference, covering notebooks 20 through 26, in the Part 1 house style: intuition first, a problem chain narrative, summary checklists, and three self checking homework problems per notebook. Highlights include quantization with FP8 and FP4 formats and a grid experiment, a runnable speculative decoding loop with measured acceptance and speedup, inference systems with batching, paging, and prefix caching simulators, an evaluation notebook using real example items from MMLU, C-Eval, CMMLU, GSM8K, and HumanEval, and a deployment notebook for serving quantized checkpoints. The project is under active development and welcomes corrections and focused pull requests. The notebook count is given as 30 or more in the badges, and the README's description mentions 26 runnable notebooks across the topics, so the precise number grows as parts are added.

Topics and teaching style

The course is organized into parts that map to the life of an LLM. Part 1 covers foundations such as tokenizer basics. Later parts move into training, then inference, then frontier topics, then production systems, as the course map shows. The teaching style is consistent across notebooks: start with intuition, build a problem chain that motivates each step, end with a summary checklist, and attach three self checking homework problems so the reader can verify understanding. The inference part rewrite is the most detailed in the README. Quantization notebook 22 covers FP8 and FP4 formats with a grid experiment and an end to end walkthrough producing GPTQ or FP8 with llm-compressor, AWQ with AutoAWQ, and GGUF with llama.cpp using imatrix, then serving each. Speculative decoding notebook 23 builds a runnable sampling loop and measures acceptance and speedup. Inference systems notebook 24 refreshes vLLM and SGLang deployment workflows with simulators. Evaluation notebook 25 walks an eval run pipeline, uses real benchmark items, maps tooling such as lm-evaluation-harness, OpenCompass, and EvalScope, explains confidence intervals, and includes a lab that registers a custom Chinese benchmark into lm-eval via YAML and scores GPT-2 against Qwen2.5-0.5B. Deployment notebook 26 serves quantized checkpoints and ties back to the pre launch evaluation checklist. This structure keeps each notebook self contained but connected to the others.

How to use it

The README offers several entry points. A reader can start in Colab with a linked notebook, read online on the GitHub pages site, or follow the English or Chinese README. There is also a Discord link for discussion. The environment is Python 3.9 or later with PyTorch 2.0 or later, and the badge shows 30 or more notebooks. Because the material is Jupyter Notebooks, the expected workflow is to open a notebook, run the cells, read the explanations, and do the homework before moving on. The course is explicitly under active development, and the Chinese course is treated as the source edition, so the English mirror may lag slightly on new notebooks. The README invites corrections, suggestions, and focused pull requests, which suits a teaching project that improves through community fixes. A quality workflow badge shows automated checks run on the repository, suggesting notebooks are at least validated for execution where the CI covers them. For someone planning to work through it, the practical path is to clone the repository or open it in Colab, pick Part 1, and proceed in order, since later parts assume the foundations. The bilingual docs and the online render make it accessible without a local Python setup, while the Colab link lets a beginner run the first tokenizer notebook immediately. The source, issues, and releases are on the walkinglabs modern-llm-notebook GitHub repository.

Editorial conclusion

The course ships as Jupyter Notebooks and its source lives at the walkinglabs modern-llm-notebook repository, with an online version on GitHub pages.