diy-llm: A Chinese-Language Rebuild of CS336 With Six Graded Assignments
Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐，6 项渐进代码作业，掌握 LLM 全栈知识
At a glance
- What is it?
- datawhalechina/diy-llm is a Jupyter Notebook course that follows the Stanford CS336 syllabus from tokenizer to GRPO, with Chinese documentation and six coursework directories. It is a curriculum, not a library, and the README does not state a licence.
- Who is it for?
- Adopt diy-llm if you already write PyTorch and want a structured, Chinese-language path through the same material Stanford CS336 covers, from BPE to GRPO, with six assignments to submit against. Do not adopt it if you need a maintained Python package, if you cannot read Chinese and cannot work from docs/en/, or if you require a stated licence before shipping derived code.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 5 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What diy-llm Is, and What It Is Not
This is a teaching repository, not a library you import. The README describes it as a Chinese-language take on Stanford CS336 (Spring 2026), and states the intent plainly: the authors want it to be more than a translated edition, calling it an "LLM alchemy workshop" built for Chinese learners. The material lives in two parallel trees. docs/zh/ holds sixteen chapters plus a preface, and docs/en/ holds an English track. coursework/ holds six assignment directories, from assignment1-basics through assignment6-evaluation. A VitePress configuration under docs/.vitepress/ renders the whole thing as a site, which is what the homepage URL points at.
The audience is narrow and the README says so. Prerequisites are listed as Python and software engineering, PyTorch and neural network fundamentals, linear algebra, probability, calculus, and machine learning. CUDA knowledge is marked optional, with the note that the project includes introductory material for readers who lack it. If you are looking for a pip install and a working checkpoint, this is the wrong repository. If you want to build the components yourself and be graded on it, the structure is unusually explicit about which chapter feeds which assignment.
The Six Assignments and How They Map to the Chapters
The coursework table is the most concrete part of the repository. Assignment 1 (手搓大模型, roughly "hand-build a large model") asks you to implement a tokenizer, model architecture and optimizer, then train a minimal language model. Assignment 2 covers performance profiling and benchmarking, a Triton implementation of FlashAttention-2, and distributed training code. Assignment 3 covers component-level understanding of the Transformer and fitting a scaling law to predict how a model will behave as it grows. Assignment 4 converts raw Common Crawl data into a pretraining dataset with filtering and deduplication. Assignment 5 applies SFT and reinforcement learning, naming GRPO, to train a model on math problems. Assignment 6 uses lm-evaluation-harness and evalscope across language understanding, commonsense reasoning, code and math.
The chapter-to-assignment mapping is not one-to-one and the README does not pretend otherwise. Assignment 1 is attached to both chapter 2 (tokenizer) and chapter 4 (architecture and training details). Assignment 2 is attached to chapters 6, 7 and 8: GPU optimization, GPU high-performance programming, and distributed training. Assignment 6 is attached to chapter 10 (inference) and chapter 12 (evaluation). That overlap is a design signal. The assignments are integrative checkpoints rather than per-chapter exercises, so you cannot finish one by reading a single chapter.
The Chapter Sequence, from BPE to Verifiable Rewards
The sixteen chapters are listed with per-chapter status markers. Chapter 1 (tooling: W&B experiment tracking, hyperparameter search, visualization dashboards) is marked as pending improvement. Chapters 2 through 15 are marked complete. Chapter 16 is marked as in progress and contains contributed material, including a piece titled "What is LLM inference?" and a section on the future of LLMs attributed to Lecun, plus three other named contributors.
The technical coverage is dense. Chapter 2 covers BPE, Unicode normalization and hand-written tokenizer training code. Chapter 4 lists RoPE, RMSNorm, SwiGLU, AdamW, Pre-Norm versus Post-Norm, and learning rate scheduling. Chapter 5 covers Top-K routing, load balancing, auxiliary loss, Expert Parallelism and DeepSeekMoE specifically. Chapter 8 lists data, model and pipeline parallelism, ZeRO-1/2/3, FSDP, gradient accumulation and All-Reduce. Chapter 9 covers the Chinchilla law, compute-optimal configuration and extrapolation. Chapter 10 lists KV Cache, speculative decoding, GPTQ and AWQ quantization, PagedAttention and continuous batching. Chapter 14 covers GRPO, rule-based verifiers, outcome and process rewards, and RLVR.
That list is a syllabus, not an implementation. The README does not state which of these have runnable reference code inside the chapters versus prose explanation, and I have not run any of it. Treat the chapter headings as the scope of the reading, and the coursework directories as the scope of the code.
Getting Started: Clone, Read, Then Pick an Assignment
The quick start is three lines. The README gives:
git clone https://github.com/datawhalechina/diy-llm.git cd diy-llm
Then a comment noting that base dependencies should be installed according to the specific assignment's needs. There is no top-level requirements.txt, no pyproject.toml and no environment.yml named in the material. The learning path is stated as three steps: read docs/zh/ in order, complete the six assignments under coursework/, then read the implementation code to understand each component's design.
The absence of a single environment file is the first practical obstacle. Assignment 2 involves Triton and FlashAttention-2, assignment 4 involves Common Crawl processing, and assignment 6 involves lm-evaluation-harness and evalscope. Those three dependency sets do not overlap cleanly, and the README's instruction to install per assignment is the honest answer rather than a missing step. Expect to create a separate environment per assignment directory. The PDF build is distributed through the releases page rather than the repository, and the README notes that a Datawhale watermark was added to discourage resale of the material.
The Licence Is Not Stated, and That Matters
The repository metadata supplied here lists the licence as unknown. The README does not name one either. For a course you read in a browser this is mostly academic. For a course where assignment 1 asks you to write a tokenizer and a model architecture, and assignment 5 asks you to run SFT and GRPO, it is not. Code you write yourself is yours, but any reference implementation you copy from coursework/ carries whatever terms the repository is under, and those terms are not visible in the material I have.
This is not a reason to avoid the project. It is a reason to resolve the question before you depend on it. The repository is under the datawhalechina organization and the README refers to Datawhale as an open source project, which suggests a permissive intent, but intent is not a licence text. If you plan to reuse coursework code in anything you ship, ask the maintainers directly. I am not giving legal advice here, only noting that the metadata field is empty and the README does not fill it.
Where It Is the Wrong Tool
Two failure modes are visible from the structure alone. The first is hardware. Assignment 2 asks for distributed training code and a Triton FlashAttention-2 implementation. Assignment 5 asks you to train a model with SFT and GRPO. The README's own localization section acknowledges that it accounts for "the computing resources at hand" and the domestic network environment, and says the project favors Qwen, DeepSeek and other Chinese models in its examples. That is a mitigation, not a solution. The assignments still require GPUs, and the README does not state minimum hardware anywhere in the material I have.
The second is language. The project is Chinese-first. The README's badge row marks the language as Chinese, docs/zh/ is described as the default, and the English track lives in docs/en/. The README_EN.md file exists, but the chapter table and assignment table are written in Chinese, and the chapter filenames are Chinese. If your team cannot read Chinese, you are relying on the English documentation being complete, and the material does not tell you whether docs/en/ mirrors all sixteen chapters or a subset. That is worth checking before you plan a study schedule around it.
Compared With the Original CS336 and With minGPT
The most direct alternative is Stanford CS336 itself, which the README names as the source. The difference is not the syllabus, which diy-llm follows closely enough to reuse the chapter ordering and the assignment structure, but the surrounding scaffolding. CS336 is delivered in English with Stanford's own lecture schedule and infrastructure. diy-llm replaces that with Chinese prose, an explicit prerequisite list, a per-chapter status table, and a stated intent to cover the prerequisite math and deep learning material so the learning curve is smoother. It also adds what CS336 does not: chapters on data engineering (MinHash deduplication, PII handling, data curriculum) and on verifiable-reward RL with GRPO, plus a chapter on multimodal models running from CLIP to Qwen-VL.
A different kind of alternative is a minimal implementation repository in the style of minGPT or nanoGPT. Those give you a few hundred lines of working training code and let you read it in an afternoon. diy-llm gives you a sixteen-chapter reading track and six assignments that expect you to produce the code yourself. If your goal is to have a working small model by Friday, the minimal-implementation route is faster. If your goal is to be able to explain why RMSNorm sits where it does and to have fitted a scaling law, the assignment structure here is the point.
Maintenance, Releases and What to Verify First
The repository is not archived and the last push is dated 2026-09-08. Two releases are listed: V0.2 on 2026-08-17 and V0.1 on 2026-06-10, roughly two months apart. That cadence suggests active work, and the chapter status table supports it: chapter 1 is marked as pending, chapter 16 as in progress, and the rest as complete. The PDF distributed through the releases page is versioned alongside the content, so an upgrade means re-downloading the PDF and re-reading any chapter whose status changed.
The upgrade cost for a learner is low, because the material is prose and notebooks rather than a dependency you pin. The upgrade cost for anyone building on the coursework code is higher, since the repository does not appear to publish a changelog in the material and the release notes are not quoted here. Before you start, verify three things: whether docs/en/ covers all sixteen chapters or only part of them, what GPU the assignment 2 and assignment 5 environments actually need, and what licence the coursework code is under. The first two you can answer by reading the repository. The third requires asking.
Editorial conclusion
Adopt diy-llm if you already write PyTorch and want a structured, Chinese-language path through the same material Stanford CS336 covers, from BPE to GRPO, with six assignments to submit against. Do not adopt it if you need a maintained Python package, if you cannot read Chinese and cannot work from docs/en/, or if you require a stated licence before shipping derived code. Before committing, open coursework/assignment1-basics/ and check that its environment file resolves on your hardware, then confirm the licence situation directly with the maintainers, because the repository metadata does not name one.
Community notes