LLMs-Zero-to-Hero: A Chinese-Language Notebook Course That Builds GPT, MoE and MLA From Scratch
从无名小卒到大模型（LLM）大英雄~ 欢迎关注后续！！！
At a glance
- What is it?
- The repository is a teaching series of Jupyter notebooks and companion videos that implement a dense GPT, a mixture-of-experts model and DeepSeek's MLA attention by hand. It is aimed at readers who want to type the code themselves rather than call an API, and its main constraint is that the material is in Chinese and still being written.
- Who is it for?
- Adopt this if you read Chinese, have a single 3090 or 4090 class GPU, and want to write attention, MoE routing and MLA projection absorption yourself instead of importing them. Skip it if you need English narration, a pip-installable library, or a finished curriculum: the README marks several rows as todo, and the chapter directories are described as still forming into a book.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 31 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap this repository is trying to fill
Most material about large language models arrives in one of two shapes. There are papers, which assume you already know why a projection matrix is absorbed into another projection matrix, and there are framework tutorials, which assume you will never look inside the model class. LLMs-Zero-to-Hero sits in between. The README states the goal as writing everything from scratch, explaining the concepts while writing the code, and it names Andrej Karpathy as the model for that style. The audience is therefore a developer who can read PyTorch and wants to see a transformer assembled line by line, not a team looking for a serving stack.
The scope is broader than a single GPT clone. The table of contents lists a dense model, an MoE model, pre-training, supervised fine-tuning, DPO, RLHF, a code-focused LLM and deployment topics such as inference optimization and quantization. The repository is a curriculum, and the README is explicit that it is incomplete. One row, the activation function chapter covering ReLU, GELU and swishGLU, has todo in both the article and notebook columns. The nanoGPT row also shows todo under the article column. Reading the table before cloning saves disappointment.
What actually runs today: the video notebooks
The concrete artifacts live under src/video/. The README links four notebooks there by name: build_gpt.ipynb, build_moe_model.ipynb, deepseek-mla-part1.ipynb and deepseek-mla-part2.ipynb. Each has a paired Bilibili video, and most have a YouTube upload as well. The MoE notebook is described as walking from a plain simplified MoE to sparse_moe and then to the share_expert_sparse_moe variant that DeepSeek uses. The two MLA notebooks split the topic: part one covers DeepSeek MLA without matrix absorption, part two covers projection absorption from both a code and a formula angle.
That split is the most interesting design decision in the repository. Matrix absorption is the step where the naive implementation and the efficient one diverge, and it is normally compressed into a paragraph in a paper. Here it gets its own notebook and its own video. A learner who only reads part one will have a correct but slow mental model, which is arguably the right order.
The repository structure block in the README shows chapter01, chapter02 and further chapter directories at the top level for written notes, with matching folders under src/ for code, plus src/hero/ reserved for the author's own model implementation. A separate repository, bit-brain, is referenced for a trainable miniLLM demo.
The hardware claim and what it implies
The README states that a 3090 or 4090 is the minimum for training, and it links a Featurize referral page advertising an RTX 4090 for 24 hours at 9.9 yuan. Treat that as the author's own environment description, not a benchmark. It tells you the notebooks are sized for a single consumer card with 24GB of memory, which in turn tells you the models being built are small. A from-scratch nanoGPT and a toy MoE fit that budget; a production-scale pretraining run does not.
This is the honest framing for the project. You are not going to reproduce a frontier model on a 4090, and the README does not claim you will. What you get is the full training loop at a scale where a forward pass finishes before you lose interest. The trade-off is that some failure modes only appear at larger scale, such as expert load imbalance in a real MoE deployment, and a small run may not surface them.
Getting the notebooks running
There is no install command in the supplied README, no requirements.txt reference and no pyproject.toml mention. The repository is a notebook collection, so the working assumption is that you clone it and open the .ipynb files, either locally or on the rented GPU instance the author points to. The README's own instructions are about following updates rather than setting up an environment: it lists a WeChat account, bbruceyuan, and a personal blog at yuanchaofa.com as the places where content appears.
Two practical notes follow from that. First, because no dependency file is documented, you should expect to resolve imports yourself inside each notebook before running it, and the notebooks are the source of truth for which packages they need. Second, the repository offers a Chinese runnable notebook badge on several rows, which signals that the notebooks are meant to be executed rather than read as static HTML.
If the from-scratch notebooks turn out to be too steep, the README points at a separate repository, Hands-On-Large-Language-Models-CN, as a gentler entry that uses transformers first. That is a deliberate on-ramp, and it is worth taking if terms like attention head and KV cache are still new to you.
The language and completion constraints
Everything in the repository is in Chinese: the README, the chapter notes, the notebook commentary and the videos. The primary language listed for the repository is Jupyter Notebook, which matches the format, but the human language is Chinese throughout. An English-speaking engineer can still read the code, since PyTorch is PyTorch, but the explanation that gives the project its value is not available without Chinese.
The second constraint is that the curriculum is a work in progress. The README says the author began writing a book in October 2025 and invites readers to follow a WeChat public account for updates. Rows in the updated-content table carry todo in the article or notebook column. The chapter01 and chapter02 directories are described as learning notes that will eventually form a book, which means their contents are expected to change. If you need a stable reference to cite in internal documentation, this is the wrong source. If you want to follow along as it grows, the update channels are listed.
A third, smaller point: the README contains promotional blocks for the author's other products, including an LLM API router and a multi-agent chat tool, plus the GPU referral link. They do not affect the code, but they are interleaved with the table of contents, so expect to scroll past them.
How this differs from reading the papers or using a framework
The obvious alternative is to read the DeepSeek MLA paper and the mixture-of-experts literature directly. That gives you the authoritative formulation, including the derivations the notebooks are reconstructing, but it gives you no runnable artifact and no debugging feedback. The second alternative is a framework such as the transformers library, which the README itself recommends as a starting point in the companion repository. With transformers you get a working model in a few lines, at the cost of never seeing where the projection absorption happens or why the expert routing is written the way it is.
LLMs-Zero-to-Hero takes a third position: the paper's content, expressed as executable cells, at a scale one GPU can handle. The cost is that you inherit the author's simplifications. A notebook-scale MoE is not a distributed MoE, and a notebook-scale MLA is not a paged-attention kernel. What you learn is the shape of the computation, which is the part that transfers.
Licence and the cost of keeping up
The repository is Apache-2.0, which permits commercial use and modification provided the licence and notices are preserved. That is a permissive choice for a teaching repository, and it means you can lift a notebook cell into your own codebase without a copyleft obligation. It does not mean the content is verified or supported; the licence covers the grant of rights, not correctness. For anything you intend to ship, treat the notebooks as a starting point and test the behaviour yourself. This is a description of the licence text, not legal advice.
Maintenance cost is the interesting question, and the material does not answer it. There are no releases retrieved, so there is no versioned artifact to pin. The repository's last push is dated 2026-08-16, and the README describes ongoing book writing, which suggests active but unversioned change. If you fork it for a course or an internal study group, expect to pin a commit hash rather than a tag, and expect that following upstream means re-reading notebooks rather than bumping a dependency. The upside of a notebook course is that it never breaks your build; the downside is that it never tells you when it has changed.
Editorial conclusion
Adopt this if you read Chinese, have a single 3090 or 4090 class GPU, and want to write attention, MoE routing and MLA projection absorption yourself instead of importing them. Skip it if you need English narration, a pip-installable library, or a finished curriculum: the README marks several rows as todo, and the chapter directories are described as still forming into a book. Before committing, open src/video/build_gpt.ipynb and src/video/deepseek-mla-part2.ipynb and check that the notebook cells run end to end on your own machine, because the repository ships notebooks rather than a pinned package.
Community notes