llms-from-scratch-cn: A Chinese-Language Notebook Course on Building GPT, Llama3, RWKV and ChatGLM From Zero

The gap this course is aimed at: architecture, not fine-tuning

The README states the project's positioning directly: fine-tuning and deployment tutorials are already plentiful, so this one concentrates on architecture implementation. That is a real gap. Most Chinese-language LLM material starts from a downloaded checkpoint and a LoRA config, which teaches you how to call a model but not why the attention block is shaped the way it is. Here the reader writes the attention mechanism, assembles a GPT class, and runs a pretraining loop over unlabelled text. The stated prerequisite is Python fundamentals plus PyTorch; the README says even with only PyTorch basics you can finish the build. The audience is therefore developers and students who want to read a transformer and recognise every tensor, not teams shipping a product. Chapter 1 has no code at all, which tells you the course expects you to read prose before opening a notebook.

Two tracks: a translated book and a bare-code path

The repository splits the fundamentals into Codes/ and Translated_Book/. The README describes Codes as the quick-entry path with concise notebooks, and Translated_Book as the detailed path with more surrounding explanation. Both derive from rasbt/LLMs-from-scratch, which the README credits by name. This duplication is deliberate but has a cost: if you read both you cover the same material twice, and if you only read Codes you skip the derivation. The chapter table is explicit about what exists. Chapter 2 covers text data handling with ch02.ipynb and dataloader.ipynb. Chapter 3 covers attention with ch03.ipynb and multihead-attention.ipynb. Chapter 4 builds the GPT model itself, shipping both a notebook and a plain gpt.py. Chapter 5 handles pretraining on unlabelled data with train.py and generate.py. Appendix A is a PyTorch primer with code-part1.ipynb, code-part2.ipynb and a DDP-script.py for distributed training. Appendix D adds training extras.

What the Model_Architecture_Discussions directory actually contains

This is the part that distinguishes the project from a straight translation. A separate directory holds per-model notebooks: ChatGLM3, Llama3, and RWKV V2 through V5, each with a named contributor and its own subfolder. The ChatGLM3 entry is titled 加载模型权重 (loading model weights), which suggests it walks through mapping an official checkpoint into the notebook's own module definitions, a different exercise from training from random initialisation. The Llama3 notebook is llama3-from-scratch.ipynb, and the RWKV guides are versioned separately as rwkv-v2-guide.ipynb, rwkv-v3-guide.ipynb and so on. RWKV matters here because it is not a transformer: it is a recurrent architecture, so reading it next to the GPT chapters shows you what the attention mechanism is buying you. The README also mentions configuration files and training scripts alongside the notebooks in this directory, though the truncated table does not enumerate them.

Running it: notebooks, the train script, and DDP

There is no package to install and no CLI. The unit of work is the Jupyter notebook, opened from a clone of the repository. For the pretraining chapter the README points at Codes/ch05/01_main-chapter-code/train.py for training and generate.py for sampling, with ch05.ipynb as the narrative wrapper and exercise-solutions.ipynb for the answers. Appendix A ships DDP-script.py if you want to move past a single device. The README does not list pinned dependency versions, a requirements.txt, or a conda environment file in the material provided, so expect to resolve PyTorch and dataset dependencies yourself. The notebooks also need the training corpora referenced in chapter 2 and chapter 5, and those are not bundled in the repository tree shown. Budget time for data acquisition before you budget time for training.

The unfinished half of the book

Three chapters are listed as 即将发布 (forthcoming): chapter 6 on fine-tuning for text classification, chapter 7 on fine-tuning with human feedback, and chapter 8 on using LLMs in practice. That is not a minor omission. It means the repository currently teaches you to build and pretrain a small GPT, and stops before it teaches you to adapt one to a task. Anyone arriving with the phrase 微调 in mind will find the chapter table pointing at empty rows. The README also notes the educational intent explicitly: the method trains small but functional models, and the README compares the approach to how large foundation models are built without claiming equivalence. Treat the pretrained output as a learning artefact, not a usable assistant.

Where a video course or a framework tutorial wins instead

If your goal is a working chatbot rather than an understanding of one, a framework tutorial is the shorter path: you load a pretrained checkpoint through a library, attach a LoRA adapter, and evaluate. That approach hides the architecture, which is exactly the trade this project makes in the opposite direction. A video course is the other alternative, and it wins on setup friction because the instructor's environment is already resolved. The difference in approach is that this repository hands you a notebook and expects you to execute cells in order, debug shape mismatches, and read the surrounding prose when a cell fails. That is slower and it is the point. The Model_Architecture_Discussions notebooks sit between the two: they load real weights, so you get a working model, but you still type out the module definitions that receive those weights.

Maintenance, licensing and what the repository does not promise

The repository is not archived and the last push recorded is 2026-03-26, so it is being touched. There are no releases, which is normal for a notebook course: there is nothing to version and nothing to upgrade. Your maintenance cost is therefore not the project's, it is your environment's. PyTorch API changes will break cells over time, and because there is no lockfile in the material provided, a notebook that ran a year ago may not run today without edits. On licensing, the README badge declares Apache 2.0 for code and links LICENSE.txt, while the repository metadata reports NOASSERTION. That discrepancy is worth resolving before you reuse notebook code in your own project, especially given the translated-book track, which is derived from another author's work and may carry separate terms for the prose. This is not legal advice; read LICENSE.txt and the upstream repository's terms yourself.

Editorial conclusion

Adopt this if you want to read transformer internals in Chinese and type out a GPT implementation yourself, and if chapter 5 is as far as your interest goes. Do not adopt it as a fine-tuning or deployment course: chapters 6 through 8 are listed as forthcoming, so the SFT, RLHF and inference chapters do not exist yet. Before cloning, open Codes/ch05/01_main-chapter-code/train.py and Model_Architecture_Discussions/llama3/llama3-from-scratch.ipynb and confirm the dependencies and weight-loading steps match the hardware you have.

Official sources

1. datawhalechina/llms-from-scratch-cn on GitHub
2. Issues
3. README