datawhalechina/base-llm: a from-scratch NLP to LLM curriculum in Jupyter notebooks

The gap base-llm is aimed at: API users who never learned the layer below

The README states the project's premise directly: many developers now go straight to calling APIs or fine-tuning large models while skipping the natural language processing foundations underneath. The stated audience is four groups: students building a base for research or job hunting, algorithm engineers moving from classical machine learning or deep learning into large models, enthusiasts who want the architecture rather than the endpoint, and researchers who want a quick refresher on classic algorithms or baseline code. The prerequisite list is explicit and worth taking literally. You need working Python, basic PyTorch experience, an understanding of neural networks, backpropagation and the training loop, and enough linear algebra, probability and gradient descent to follow the notation. This is not an entry point for someone who has never trained a model. The project's own framing is that it fills a gap between two kinds of material: tutorials that stay academic and opaque, and tutorials that stop at the API call. Whether it succeeds depends on the code, not the framing, and the code is what the chapter list exposes.

How the curriculum is sequenced, and why the ordering is the real design decision

The table of contents is organized into six parts plus an Extra-chapter area. Part one is theory: NLP overview and environment setup, tokenization and word vectors including Word2Vec and a Gensim walkthrough, RNN with LSTM and GRU, Seq2Seq and attention leading into Transformer, then pretrained models (BERT, GPT, T5, Hugging Face), then large model architecture covering a hand-written Llama2, MoE, text generation strategies and in-context learning. Part two is applied work on text classification and named entity recognition. Part three is fine-tuning and quantization: PEFT, LoRA, a peft-based LoRA exercise, QLoRA fine-tuning of Qwen2.5 on private data, RLHF with a LLaMA-Factory DPO exercise, model quantization and DeepSpeed. Part four is deployment: FastAPI serving, a cloud server deployment walkthrough using uv on Linux, Docker Compose, then Git and a Jenkins CI/CD pipeline. Parts five and six cover safety and multimodal topics. The sequencing claim in the README is that each step answers why the previous one was insufficient, for example why RNN gave way to Transformer. That is a pedagogical choice with a cost: a reader who only wants LoRA fine-tuning has to decide how much of parts one and two to skip, and the repository does not mark optional paths.

Code evolution as the teaching method, and what that means for readers

Two of the stated highlights are that the material avoids being a pure library-calling exercise and that it uses a propose-then-refactor pattern, showing the movement from a simple script to something closer to an industrial framework. The clearest instance named in the outline is chapter 6's handcraft_llama2 notebook, described as building a large model by hand. Chapter 6 also includes a notebook on text generation strategies and one on MoE architecture. In the applied half, the NER section is split into data processing, model building and training, and evaluation and prediction, which is the shape of a real project rather than a single demonstration. The trade-off here is that incremental refactoring produces notebooks that are long and stateful. If you read them out of order, or run cells selectively, you lose the thread of which version of the model is current. The repository is a set of teaching artifacts, and treating it as a maintained codebase with stable internal APIs will lead to confusion. The README's own contribution note reinforces this: pull requests are currently accepted mainly for the Extra-chapter area, with a pointer to Extra-chapter/README.md for the rules.

What the repository does not tell you: licence, release cadence and unfinished parts

The repository metadata supplied does not declare a licence. That is a concrete blocker if your plan involves copying notebooks, adapting the Llama2 implementation, or shipping anything derived from the code. Without an explicit repository licence, reuse permissions for copying, adaptation and redistribution are unclear; platform terms and applicable law may also matter. Check the current repository and ask the rights holders before relying on broader reuse rights. Ask before you build on it. The release history is thin: one recent release, v2026-pdf-beta, dated 2026-03-08, described as a beta PDF publication. Pinning your own dependency versions is on you. The outline also shows two chapters marked as under construction under the safety section, behavior alignment engineering and safety architecture design, with unchecked boxes. The multimodal section, by contrast, lists completed notebooks including a simplified Omni model trained from scratch. If your interest is alignment engineering, the curriculum currently stops short of it.

Getting the material running: the entry points that actually exist

The primary access path is the online book at https://datawhalechina.github.io/base-llm/, and the README also publishes a v2026-pdf-beta release for offline reading. For the notebooks, clone the repository and work from the docs directory, where the outline maps each chapter to a file path such as docs/chapter1/01_nlp_intro.md, docs/chapter6/17_handcraft_llama2.md, docs/chapter11/04_qwen2.5_qlora.md and docs/chapter14/03_docker_deploy.md. The environment chapter is docs/chapter1/02_preparation.md, and the badge in the README states Python 3.10 or later. The deployment chapters name their own tools: FastAPI for serving, uv and Linux for the cloud server walkthrough, Docker Compose for multi-service deployment, and Jenkins for the CI/CD pipeline exercise. The fine-tuning chapters name peft for LoRA and LLaMA-Factory for the DPO exercise, with DeepSpeed covered separately under training and quantization. Note the file naming: one chapter file contains an ampersand, docs/chapter3/09_LSTM&GRU.md, which will need quoting in a shell. The notebooks are the source of truth for versions.

Where base-llm is the wrong tool

Three cases stand out. First, if you need a production library rather than a course, this is not it: the artifacts are notebooks and Markdown, the contribution policy is scoped to Extra-chapter, and there is no published release cadence beyond the PDF beta. Second, if you cannot read Chinese, the value drops sharply, because the prose is the teaching surface and the code alone does not carry the reasoning. Third, if your goal is alignment or safety engineering specifically, the two chapters you want are listed as under construction. There is also a hardware reality the outline implies but does not resolve: QLoRA fine-tuning of Qwen2.5 on private data, RLHF with LLaMA-Factory, and DeepSpeed training all have memory requirements that vary enormously by configuration. Verify that against the notebooks before assigning them to a team on laptops. The honest summary is that base-llm is strong as a structured reading path and weak as an infrastructure dependency.

Alternatives and the difference in approach

The most direct comparison inside the same ecosystem is Hugging Face's own course material, which base-llm itself references as a topic via chapter 5's HuggingFace notebook. The difference is orientation. Hugging Face course content is organized around the transformers, datasets and tokenizers libraries, so you learn the API surface and the workflows those libraries support, and the code you write is code you would keep. base-llm inverts that: it spends its first four chapters on tokenization, word vectors, RNN, Seq2Seq and attention before the libraries appear, and it asks you to write a Llama2 implementation by hand in chapter 6. The cost is time and the reward is mechanism-level understanding, which matters when you have to debug a shape mismatch or explain why a generation strategy produces the output it does. A second comparison is the DeepSpeed and LLaMA-Factory documentation themselves, which are the upstream sources for base-llm's training and DPO chapters. Those documents assume you already know what you are training and why; base-llm supplies the why, and they supply the current flags and supported configurations. Using base-llm for concepts and upstream docs for exact arguments is the combination that avoids stale command lines.

Maintenance cost and the licence question you must resolve first

Treat base-llm as a reading commitment rather than a dependency you upgrade. The maintenance burden lands on you: record the library versions your environment resolves when a notebook first runs, and expect the DeepSpeed, peft and LLaMA-Factory chapters to drift as those projects change. The single release listed, v2026-pdf-beta, is a documentation artifact, not a code release, which tells you the project's output is the text. On licensing, the metadata shows no licence, and this is not a legal question I can answer for you: the practical step is to open an issue on the repository asking the maintainers to declare one, and before redistribution or incorporation into a product, obtain the necessary permission or qualified legal advice. That is a real constraint on adoption inside a company, and it is better discovered now than after a team has built on the notebooks.

Editorial conclusion

Adopt base-llm if you are a Python-fluent developer with PyTorch experience who wants the RNN to Transformer to LoRA chain explained in sequence and is willing to read Chinese. Do not adopt it as a reference implementation to vendor into a product: the licence is undeclared in the repository metadata, so you have no stated grant to reuse the code. Before starting, open the online book at datawhalechina.github.io/base-llm and read chapter 6's handcraft_llama2 notebook plus chapter 11's qwen2.5_qlora notebook end to end, since those two files decide whether the practical half of the curriculum matches your hardware and your framework versions.

Frequently asked questions

Official sources

1. datawhalechina/base-llm on GitHub
2. Issues
3. Project website
4. README
5. Releases