llms-from-scratch-cn: two notebook trees, three missing chapters, and one licence question

Two notebook trees cover the same five chapters

The root of the repository holds Book/, Codes/, Model_Architecture_Discussions/, Translated_Book/, images/ and .gitignore, and two of those trees teach the same chapters at different depths. Codes/ is the fast path: the README points beginners there because the code is kept short. Translated_Book/ is the long path, holding notebooks with more explanation for the same material. The reader picks a depth once rather than assembling a path chapter by chapter.

Each chapter directory follows a similar internal shape, a numbered main-chapter-code folder (01_main-chapter-code for chapters 2 to 5, 03_main-chapter-code for appendix A) plus supporting material at the chapter level. Chapter 2 on text data splits into ch02.ipynb, dataloader.ipynb and exercise-solutions.ipynb. Chapter 3 splits into ch03.ipynb and multihead-attention.ipynb. So a chapter is rarely one file, which matters if you were expecting a script you can run top to bottom.

The claims of full coverage collide with three chapters marked as coming soon

One passage says the project contains all the code for a GPT-like model, covering tokenisation, pretraining and fine-tuning. The chapter table three paragraphs below it lists chapter 6 on fine-tuning for text classification, chapter 7 on fine-tuning with human feedback and chapter 8 on using LLMs in practice as 即将发布 in both the main-code and the all-code columns. The same rows say 没有代码 for chapter 1, so the table is distinguishing absence by design from absence by schedule.

That gap is deliberate in part. The README says that with plenty of tutorials on fine-tuning and deployment already available, the project concentrates on architecture implementation. If your goal is instruction tuning on your own data, this repository does not take you there.

Architecture guides are pinned one notebook per model version

Model_Architecture_Discussions/ does not hold one guide per family. It holds one notebook per checkpoint: ChatGLM3, Llama3, RWKV V2, RWKV V3, RWKV V4, RWKV V5, RWKV V6, ChatGLM4 and MiniCPM. Five of those nine are RWKV versions, so the folder spends more of its weight on one recurrent architecture than on the decoder-only stack the book chapters build.

Each row in the README's architecture table also carries a contributor. One contributor is credited with RWKV V2 through V6 plus ChatGLM4, another with ChatGLM3 and another with Llama3. Per-contributor ownership is the quality signal the architecture table shows.

There is no install command to copy, and no requirements file at the root

The root listing carries no requirements.txt. The only entry point it offers is the chapter table of links, which opens notebooks directly on GitHub or on the AtomGit mirror at atomgit.com/datawhalechina/llms-from-scratch-cn. That is where to get the notebooks, and it is also where to find out that nothing here gets installed.

In practice the reader assembles the environment by hand. The stated prerequisite is PyTorch familiarity, not a fixed stack, so the version of torch a notebook assumes is whatever the reader happens to install that week, and the last push was on 2026-09-17, and licence detection returns no assertion even though a LICENSE.txt sits in the root, so read that file before you reuse a notebook.

For a course you intend to follow over weeks, writing down your own torch version on day one is the difference between a reproducible notebook and a debugging session. The same caution applies to the AtomGit mirror, which is a second copy.

Chapter 4 hands you a plain gpt.py, chapter 5 hands you train.py and generate.py

The scripts that escape the notebook are the ones worth knowing about. Chapter 4 pairs its notebook with a standalone gpt.py, and chapter 5 adds train.py for the pretraining loop and generate.py for sampling. These are the files to read when you want the architecture without the surrounding narration.

The naming is inconsistent enough to matter when you search. Chapter 2 and chapter 3 keep their helper work inside dataloader.ipynb and multihead-attention.ipynb, chapter 4 moves the model into gpt.py, and appendix A uses code-part1.ipynb and code-part2.ipynb. Nothing maps one convention onto the other.

Distributed training appears once, in appendix A's DDP-script.py

Appendix A covers a PyTorch introduction, and its only file that is not a notebook is DDP-script.py. That placement is informative. It is the only file in the chapter table whose name points at distributed training.

Two appendix entries carry no code at all. Appendix B on references and appendix C on exercises are marked 没有代码, while appendix D on adding extra features to the training loop is a single notebook. An exercise set with no solutions attached means the reader has nothing to check against except the exercise-solutions.ipynb files that chapters 2 through 5 and appendix A do ship.

Where this course sits against rasbt's original and against loading pretrained weights

The credits name rasbt/LLMs-from-scratch as the basis for the fundamentals section, and Sebastian Raschka is thanked directly. The difference is additive rather than competitive. This repository builds on that work, then adds the Model_Architecture_Discussions folder for GLM4, Llama3 and RWKV6, which the book chapters never cover.

Against a pretrained-weights library the difference is one of intent. Loading ready weights skips the chapters 2 through 5 exist for, since tokeniser, data loader, attention, model, training loop and generation are the objects under study. This course asks you to write each of those yourself. The cost is environment drift and the missing later chapters; the return is that you can explain what sits between a token and a logit.

Editorial conclusion

Adopt this repository if you are learning LLM internals in Chinese and want chapter-by-chapter notebooks plus a standalone gpt.py you can read end to end. Do not adopt it if you need fine-tuning, human-feedback training or deployment code, because the chapter table still marks chapters 6, 7 and 8 as 即将发布, and no exercise solutions are attached to appendix C. Verify first what LICENSE.txt actually grants, since licence detection reports nothing for this repository and the notebooks are the deliverable here.

Frequently asked questions

Official sources

1. datawhalechina/llms-from-scratch-cn on GitHub
2. Issues
3. README