bojieli/ai-infra-book: A Quantified AI Infrastructure Textbook and Its Calculation CLI

The gap this manuscript targets: numbers before architecture

The README frames the motivation as a missing book: AI infrastructure lacks a text that starts from hardware constraints and model architecture and derives system design quantitatively, in the way Computer Architecture: A Quantitative Approach does for processor design. The stated audience is engineers who have already called a model API or run a model locally and now want to know why it is slow and how to cut cost. The README also names three narrower tracks: model serving and applications (chapters 8, 9, then 11 and 12), systems and networking (chapters 5 to 7, then 9 and 10), and chips and architecture (chapters 4 to 7). Prerequisites are stated as some Python, linear algebra and computer systems background, with the preface carrying the detail. This is not an operator's manual for a serving framework. It is an estimation discipline: list compute, storage, communication and dependencies, compare them against hardware capacity, bandwidth and throughput, then check how far a measurement sits from the physical ceiling. The README is explicit that the book is a draft still being revised, so treat chapter content as provisional.

From constraint to design: the five questions the book keeps asking

The method described in the README is a repeated interrogation of data movement: what is moved, how much, how many times, along which path, and who has to wait for it. The author traces that thread through FPGA-accelerated Bing search ranking, Ascend AKG operator generation and UB ten-thousand-card interconnect. The failure modes the README calls out are concrete and familiar to anyone who has done a back-of-envelope estimate: counting weight reads but forgetting the KV cache, extrapolating from peak compute without checking whether bandwidth can feed it, splitting work across cards while ignoring inter-card communication. Any one omission moves the answer by a factor, sometimes by orders of magnitude. That framing is the book's real contribution as a repository artefact, because it turns into a checkable procedure in the companion calculator rather than staying as advice. Chapter 1 poses the entry question directly: how much memory, compute and data read/write does a single generation need. Chapters 2 and 3 then vary architecture and workload before the hardware chapters arrive.

The twelve-chapter structure and what each block assumes

The table of contents runs from chapter 1 (what AI infrastructure is, framed by the memory, compute and I/O cost of one generation) through model architecture, inference and training workloads, accelerator architecture, operators and runtime, supernodes, data center networks, inference optimization, distributed inference, training systems, resource scheduling and runtime environments, and finally device-edge-cloud coordination. The ordering matters: chapters 4 to 7 establish hardware and interconnect limits, and the later chapters spend those limits on batching, KV management, offloading, speculative decoding, placement, scaling and recovery. Chapter 11 addresses how model serving and tool environments share resources and reduce waiting, which is the part closest to day-to-day platform work. Chapter 12 closes on where a task should run, local, edge or cloud, weighing quality, latency and cost. The README recommends reading chapters 1 to 3 first regardless of track, because the workload vocabulary is used everywhere afterwards. The README also suggests estimating each worked example yourself before reading the derivation, and substituting your own model and hardware to see whether the conclusion survives.

The calculation CLI is the part you can actually run

The repository ships a static calculator under calculations/ that needs only Python 3.10+ standard library, with no GPU and no model weights. The README gives the exact sequence: clone the repository, change into it, then list supported models with python3 calculations/calc.py models, and estimate per-operator resource needs for Qwen3-8B at an 8192-token prefill with python3 calculations/calc.py forward --model qwen3-8b --tokens 8192 --format md. The tool carries model configurations and a results index under calculations/results/, so the numbers printed in the book can be recomputed and, more usefully, re-derived for a different model and input shape. The --format md flag implies other output formats exist, though the README shows only this one. This is the strongest adoption argument in the repository: the estimation method is executable, so a disagreement with the book becomes a runnable counterexample rather than a forum argument. It is also the piece most likely to be extended by readers, since the README lists fixing bugs in experiments/ and calculations/ among the wanted contributions.

Cloning, Git LFS and the 20 GB you probably do not want

The repository stores papers and larger input and measurement records in Git LFS, roughly 20 GB in total. The .lfsconfig file skips all LFS files by default, so a normal clone leaves pointers in the working tree; the manuscript, figures and static calculations do not need them. To reproduce a specific experiment you fetch only that directory, and the README gives the pattern: git lfs install, then git lfs pull --include="experiments/ch05/05-01/**" --exclude="". Experiments live under experiments/chXX/XX-YY/ with run instructions, input conditions and result notes, and GPU-dependent ones state their hardware and dependency requirements. The README advises recording the manuscript version, model, hardware and input parameters when reproducing or citing a result, which is the right instinct for a document that is rebuilt several times a day. If you only want to read, the default clone plus the PDF link is enough; the LFS dance is for replication.

Building the site and the PDF yourself

Two build paths are documented. For the reading site, the README uses a virtual environment: python3 -m venv .venv-site, source .venv-site/bin/activate, python -m pip install -r website/requirements.txt, then python scripts/build_site.py and python scripts/check_site.py, with python scripts/build_site.py --serve for a preview at http://127.0.0.1:8000 and output under build/. For the full PDF, bash book/build_pdf.sh requires Pandoc, XeLaTeX and fonts, with per-chapter compilation described in book/README.md. GitHub Actions checks both the site and PDF builds on pull requests, and pushes to main produce a release and deploy Pages. The README is blunt about reading format: formulas, tables, footnotes and cross-references render incompletely or misaligned when GitHub displays the Markdown, so the PDF, typeset with XeLaTeX and republished on every main update, is the recommended way to read the book end to end. The Markdown under manuscripts/ remains the single source for both outputs, which is why corrections should be sent there rather than to the generated site.

Where it stops being the right tool

Three limits are visible in the repository itself. First, the README states the manuscript is a draft under continuous revision, so chapter text and numbers can move between builds; the release identifiers are timestamped (build-20260914-145529 and similar), which tells you the artefacts are regenerated frequently but not that any given figure is settled. Second, the calculator is static: it estimates from model configurations and inputs, and the README does not claim it models runtime contention, scheduler behaviour or real interconnect congestion, so it will not predict the throughput of your specific cluster. Third, experiment reproduction depends on hardware that the README says is noted per experiment; without the matching GPU you are reading recorded results, not verifying them. There is also no documented rollback or version-pinning procedure for the calculator CLI, and no stated compatibility promise across releases. If you need a supported reference for a production capacity plan, this is a draft textbook plus a research calculator, not a vendor specification.

Alternatives and the difference in approach

The obvious comparison is the author's earlier book, 深入理解 AI Agent, described in the README as the companion volume and the source of the ElegantBook/XeLaTeX template reused here. That book starts from the model as an application component: agents, design principles and engineering practice. This one starts one layer down, from hardware capacity, bandwidth and compute, and derives what the system above can do. A second reference point is framework documentation, such as a serving stack's own performance tuning guide. Those documents describe the knobs a specific runtime exposes and how to set them; this repository does not document any runtime's knobs. It gives you the arithmetic to decide whether a knob is worth turning at all, and the calculator to check the arithmetic. If your question is "which flag do I set for continuous batching in my server", the framework docs answer it. If your question is "can this hardware configuration serve this model at this context length, and which resource runs out first", that is the question this repository is built around. The two are complementary, and the README's chapter routing assumes you already have a runtime in hand.

Maintenance, licensing and the cost of staying current

The repository is not archived, and the last push was on 2026-09-14, the same day as the most recent release, so the build pipeline is running now. The release list shows three full-book builds within roughly two days, which indicates that the site and PDF are regenerated on every main update rather than at chapter milestones. That is good for freshness and bad for citation stability: a page number or a computed value you quote can shift under you, which is why the README asks readers to record the manuscript version, model, hardware and input parameters alongside any reproduced result. Upgrade cost is therefore low in tooling terms (a git pull and a rebuild) but non-trivial in review terms, because each pull can change numbers you have already circulated. On licensing, the README states that the original text, figures and companion code are Apache-2.0, Copyright 2026 Bojie Li, while third-party code, fonts, templates and reference material keep their own copyright and licence terms and are not relicensed by inclusion. The practical consequence is that reusing the prose or figures is straightforward under Apache-2.0 with attribution, but the LFS-stored papers and any bundled third-party assets need their own licence check before redistribution. This is a description of what the repository states, not legal advice.

Editorial conclusion

Adopt it if you already call model APIs or run models locally and want order-of-magnitude reasoning about memory, compute, bandwidth and communication before you tune a serving stack; the calculator runs on Python 3.10+ standard library with no GPU or weights. Do not adopt it as a production reference: the README states the manuscript is still a draft under revision, and the repository does not document a deprecation policy, rollback guidance or a compatibility guarantee for the calculations CLI. Verify first that the model you care about is listed by python3 calculations/calc.py models, then recompute one chapter example with your own token count and hardware constants before trusting any number in the PDF.

Official sources

1. bojieli/ai-infra-book on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases