100-days-of-inference: A Notebook Curriculum Tied to One Book

A Syllabus, Not a Library

The repository does not ship an inference engine. It ships a plan for learning how inference engines work, organized as daily entries that the README describes as runnable scripts. The distinction matters at the point of adoption: there is nothing to import, no server to start, no config file to point at your model. What you get is a sequence of topics with a mapping to specific chapters of Inference Engineering by Philip Kiely, published by Baseten Books. Days 01 through 18 are listed with links (./day01/ through ./day18/), each carrying a topic and a chapter reference such as Ch 2.2 for LLM inference mechanics or Ch 4.3.1 for vLLM and PagedAttention. Phase 2 continues the pattern for days 19 through 26, covering GPU architecture, MIG, Docker, autoscaling, routing, multi-cloud capacity and zero-downtime deployment. The audience is narrow and clear: one engineer, working alone, who already knows Python and wants to understand what happens between a prompt and a token. Anyone looking for a maintained package has opened the wrong repository.

Phases 1 to 3 Follow the Book's Three Layers

The README quotes Kiely's framing that doing inference well requires three layers: Runtime, Infrastructure and Tooling. The phases mirror that split. Phase 1 is single-instance optimization and runs from tokenization through attention, the KV cache, the ops:byte ratio, CUDA kernels, model formats, then the serving runtimes (vLLM, SGLang, TensorRT-LLM, NVIDIA Dynamo), then quantization, speculative decoding, prefix caching, parallelism and prefill/decode disaggregation. Phase 2 moves outward to the cluster: SMs and HBM, GPU generations, MIG partitioning, containers, autoscaling, routing and multi-cloud capacity. Phase 3 is the instrumentation layer, listed as three topics without day numbers: performance benchmarking and profiling, observability with metrics and tracing, and client code covering streaming, async and protocol support. The ordering is deliberate. You cannot evaluate a PagedAttention block allocator before you understand why the KV cache exists, and the README puts the KV cache on day 05, well before vLLM on day 09.

Phases 4 to 6 Turn Chapters into Build Tasks

The second half of the README stops linking to directories and starts listing project titles instead. Phase 4 is titled Deep Implementation and reads as a from-scratch exercise list: a BPE tokenizer, a bare autoregressive decoder loop in PyTorch, scaled dot-product attention with masking, a simplified tiling implementation of Flash Attention in Python, an INT8 quantize-dequantize-error pipeline, GPTQ-style round-to-nearest with Hessian weighting, a block allocator with an eviction policy, hash-based prefix cache deduplication, a simulated tensor-parallel matmul split across N workers, a Triton elementwise kernel, and a PyTorch custom op with a CUDA backend. Phase 5 is production systems: a vLLM Dockerfile, a NIM-compatible container, an autoscaling policy simulation, cold start measurement, round-robin and least-connections load balancers, a priority queue, a $/token cost model across instance types, blue-green model swaps, Prometheus metrics, a Grafana dashboard, OpenTelemetry tracing, Locust load testing, Nsight Systems profiling, SSE and asyncio clients, geo-aware routing, memory profiling, and a reusable benchmark harness. Phase 6 covers modalities beyond text, naming vision language models, embedding models and ASR, each with a chapter reference. The README is truncated during the ASR entry, so the full scope of Phase 6 cannot be confirmed from the supplied material.

The Hardware Assumption Is Two DGX Sparks

The README states that all experiments run on a home-lab cluster of two NVIDIA DGX Sparks, and later entries name those machines as spark-01 and spark-02. This is the single largest constraint on reproducing anything here. Multi-GPU tensor parallel benchmarks, MIG profile configuration, disaggregated prefill experiments and cross-node scaling measurements all assume that specific two-node topology. A reader with one consumer GPU can still follow the runtime phase, since attention, the KV cache, quantization error and kernel fusion are single-device topics. A reader with no NVIDIA GPU at all will find the CUDA kernel days, the Triton day and the Nsight profiling day difficult to run as written, though the README does not say whether CPU fallbacks exist. Treat the hardware line as a scope declaration rather than a suggestion. It tells you which entries are reproducible on your machine and which are read-only for you.

Getting It Running: Clone, Then Read the Day Directory

There is no install section in the README, no requirements file mentioned, no setup command. The only concrete instruction implied by the material is to clone the repository and open a day directory, for example day09 for vLLM and PagedAttention or day13 for quantization number formats. Because the project is Jupyter Notebook based, the practical entry point is a notebook server pointed at the repository root. What each notebook imports cannot be determined from the README, and the README does not list dependencies, a Python version, a CUDA version, or pinned package versions. That absence is the main operational risk. A notebook that calls vLLM, SGLang and TensorRT-LLM across different days will pull in three separate serving stacks with their own CUDA and driver expectations, and nothing in the supplied material says those are isolated per directory or reconciled in a shared environment file. If you plan to work through more than a few days, expect to build your own environment per day and to treat the notebooks as reference implementations rather than a coherent project.

What the README Does Not Tell You

Several things a reader would want before committing are missing. The license is unknown, which means the terms under which you may reuse the notebooks are not stated in the repository metadata. There are no releases, so there is no versioned snapshot to pin against and no changelog describing what changed between pushes. The last push is dated 2026-04-30, which places the repository's activity in the future relative to most readers' present; whether the plan was completed is not something the README confirms. The README itself is truncated mid-entry in Phase 6, so days beyond the ASR topic are unknown. Most importantly, the README describes Phase 4, 5 and 6 entries as project titles without directory links, unlike days 01 through 26. That asymmetry suggests the linked days exist as directories and the later phases may be a forward-looking plan rather than finished work, but the supplied material does not settle the question. Check the repository tree directly before assuming any specific phase is complete.

Compared with a Hands-On Serving Tutorial

The obvious alternative is a single-purpose serving walkthrough, of which vLLM's own documentation is the clearest example: it takes one engine, gives you an install command, a Python entry point and a set of configuration flags, and stops there. The difference in approach is breadth against depth. A vLLM tutorial gets you to a running endpoint in an afternoon and tells you nothing about why PagedAttention uses blocks, what arithmetic intensity means for your batch size, or how prefill and decode behave differently under load. This repository inverts that: it spends eighteen days on runtime internals before it asks you to deploy anything, and its deployment material is spread across simulated autoscaling policies, cost models and load balancers rather than one working server. Neither is a substitute for the other. If your goal is to serve a model this week, read the vLLM documentation. If your goal is to be able to reason about why the vLLM defaults are what they are, this sequence is the more direct route, provided you accept that you are following a curriculum rather than installing a tool.

Maintenance Cost and the Licence Gap

The maintenance burden here is personal rather than operational. Nothing in this repository runs in your production path, so there is no upgrade treadmill in the usual sense. The cost is that every notebook pins implicitly to whatever serving stack was current when it was written, and the LLM inference stack moves quickly: quantization formats, attention kernels and serving runtimes all change at a rate that makes two-year-old notebook code fragile. Because there are no releases and no dependency manifest in the supplied material, there is no upgrade path other than reading the diff between pushes. On licensing, the repository metadata does not state a license at all, which is different from stating a permissive one. Without a license file, the default position under most copyright regimes is that no reuse rights are granted, so copying notebook code into your own project carries uncertainty that a repository with an explicit MIT or Apache-2.0 header would not. This is a factual gap in the repository, not legal advice; if you intend to reuse the code, resolve the license question with whoever holds the copyright before you do.

Editorial conclusion

Adopt this if you are an individual engineer working through Inference Engineering and want a reading order plus a checklist of things to build; the repository is most useful as a syllabus, not as a dependency. Do not adopt it if you need a supported library, a pinned environment, or anything you can put in a production path. Before spending a weekend on it, open the day directories that matter to you and check whether they contain actual notebooks and whether the code imports anything beyond PyTorch, since the README describes days 01 through 26 as links while the Phase 4, 5 and 6 entries are listed as project titles with no paths at all.

Official sources

1. elizabetht/100-days-of-inference on GitHub
2. Issues
3. README