zero-to-sglang: build a mini inference engine, then read the real SGLang source

A course aimed at the gap between inference concepts and the SGLang source tree

zero-to-sglang is not a library you install. It is a course, published as Markdown under course-material/ch and course-material/eng and rendered through a VitePress site, aimed at engineers who can use an inference engine but cannot explain what it does to a request. The README states the problem plainly: existing material either stays at concept level, or drops you straight into the SGLang source with no entry point. This course takes a middle path. Part I builds the mental model, Part II has you write a mini-sglang yourself, Part III returns to the real SGLang code, and Part IV walks the contribution workflow. The project is a joint effort between Datawhale and RadixArk, which the README identifies as the company founded by the SGLang team, and the chapter table marks specific chapters as written by SGLang members. That provenance matters for a source-reading course, because the chapters on attention backends, quantization and the PR workflow are the ones that come from people who work on the engine itself.

Part I runs without a GPU and is the part you can read today

Part I is deliberately runnable on nothing. The README states it covers concepts only, with no code and no GPU, and the hardware note repeats that Part I does not need one. The chapter list gives the actual topics: training versus inference, prefill and decode, compute-bound versus memory-bound, the Roofline model, and a chapter that derives KV Cache from the attention computation and then quantifies its memory footprint. Chapter 5 covers benchmark metrics, listing TTFT, TPOT, ITL and Goodput, plus percentiles and tail latency. This is also the most finished part of the course. Chapters 1, 2 and 4 carry the complete marker, while chapter 3 on GPU architecture and chapter 5 on benchmarking are marked as updating. Anyone deciding whether to follow along can read Part I today and judge the writing quality directly, since the online version is public at the project's GitHub Pages site.

The mini-sglang build sequence doubles as a reading order for the real engine

The middle of the course is what separates it from a normal blog series. Part II plans ten chapters that each add one subsystem to a small engine you write yourself: a first pass of roughly 200 lines covering the forward computation and autoregressive generation, then KV Cache described as moving attention from O(n²) to O(n), then an HTTP layer for concurrent requests, then continuous batching with a scheduler, paged KV Cache with memory management, RadixAttention with prefix caching, multiprocess tensor parallelism, and finally speculative decoding. Every one of those names is a mechanism in the real SGLang, so the build order doubles as a reading order for the production code. The README links a reference implementation at sgl-project/mini-sglang for comparison. Two caveats follow from the material itself. All ten chapters carry the in-preparation marker, and the schedule dates their writing from 9.14 to 10.04 with review through 10.10. Today the sequence is an outline, not lessons you can follow.

Parts III and IV are a dated promise about SGLang internals

Part III covers the optimizations that make the real engine fast: attention backends including FlashInfer, Triton, FA3 and FlashMLA, together with CUDA Graph, quantization and low-precision inference, hierarchical caching, scale-out through DP Attention, EP and PP, and prefill-decode disaggregation. Part IV is a contribution track, with deploying SGLang through a cookbook, profiling and trace analysis, and a walk-through of the SGLang PR workflow. The README is unusually explicit about what is not finished. Every chapter in Parts II, III and IV carries the construction marker, and the progress table dates Part III from 10.10 to 11.15 and Part IV from 11.16 to 12.6. The course started on 2026-08-24 and plans roughly one chapter per week. The repository's last push was 2026-09-10, one day before the material's snapshot date, which fits the stated review window for Part I that runs through 9.13. Someone starting today gets a complete Part 0, a nearly complete Part I, and a promise about everything else.

Clone it, read it online, and the CPU versus GPU split

Getting the material is a single clone: git clone https://github.com/datawhalechina/zero-to-sglang.git. The README lists no dependencies and no install step beyond that, which fits a repository whose content is Markdown plus a VitePress site rather than a package. Hardware requirements differ by part. Part I needs no GPU. Part 0 already contains a chapter that deploys a real SGLang server running Qwen3-0.6B on your own GPU, so even the warm-up touches real hardware. Part II, per the README, can be debugged mostly on CPU, with a GPU recommended for the complete implementation and for performance measurement, and the text notes a cloud instance is acceptable. Language coverage is uneven. The Chinese tree holds all parts, while course-material/eng currently contains only part0 and a writing template, with the English translation marked as in progress. An English-only learner currently depends on material that has barely started to appear.

No declared license, and two alternatives that already exist

The repository declares no license; the metadata lists it as unknown. For a course this is a real constraint on reuse. Reading is one thing, but rehosting chapters or adapting them into internal training material has no granted permission, and asking the maintainers or waiting for an explicit license is the safe path. It says nothing about code you write yourself while following Part II, which is your own work. If the need is immediate rather than pedagogical, the material itself names two alternatives. One is sgl-project/mini-sglang, the reference implementation, which gives you the small engine now, minus the lessons. The other is the SGLang repository and its documentation, which answers operational questions, deployment first and configuration second, without teaching you why the scheduler looks the way it does. The course's bet is that the space between those two is where understanding actually forms, and that bet only pays off if the scheduled chapters ship.

Editorial conclusion

Engineers who want to understand SGLang internals before adopting or contributing to it should follow Part I now and judge the writing directly. Teams that need to serve models this week should go straight to the SGLang repository and its documentation, and learners who need English should check how far course-material/eng has progressed before committing study time. Verify three things first: the chapter status markers in the README, whether the Part II chapters dated from 9.14 have appeared, and whether a license has been declared, since today the repository offers a strong beginning, no license, and a schedule for everything else.

Official sources

1. datawhalechina/zero-to-sglang on GitHub
2. Issues
3. Project website
4. README