time-to-first-token: a 10-week roadmap that produces one inference service instead of 50 disconnected exercises

The problem is setup cost, not missing knowledge

Most inference learning material is a list of topics. You read about PagedAttention, then you read about continuous batching, then you read about speculative decoding, and each one arrives in its own environment with its own install steps and its own model download. The README names this failure mode directly: the alternative approach of "running seventeen disconnected experiments" spends most of its time on setup. The claim is that one service which keeps growing covers the same ground and leaves an artifact behind.

That is a real distinction. A disconnected experiment ends when the notebook stops running. A service accumulates: the endpoint you stood up in week 2 is still there in week 6 when you attach quantization, and still there in week 7 when speculative decoding goes in front of it. The audience is narrow and stated plainly. You need Python, transformers at the architecture level, and the command line. Prior serving, Kubernetes, or CUDA experience is explicitly not required, which is the useful part of the claim and also the part that deserves scrutiny later.

Why the roofline model comes before any tuning knob

The roadmap's ordering is its actual content. Five decisions drive it, and the README documents each one.

The first is that the roofline model arrives in week 1, before vLLM is even installed. The reasoning given is that every later optimization is a move on the same plot. Quantization attacks memory-bandwidth-bound decode. Continuous batching raises arithmetic intensity toward the compute roof. Speculative decoding spends FLOPs, which the README calls cheap in the memory-bound regime, to cut sequential memory loads. Disaggregation exists because prefill is compute-bound and decode is bandwidth-bound and the two fight over one GPU. Without that frame, the README argues, the rest is a bag of tricks.

Week 1's deliverable is a hand-derived arithmetic intensity figure for the model you intend to serve, plus the ability to say which phase is bandwidth-bound. That is a concrete output, and it is the kind of thing you cannot fake by watching a lecture. The reading is Horace He's "Making Deep Learning Go Brrrr From First Principles" split across two days, then the first and second halves of Stanford CS336 Lecture 5 on GPUs, with kipply's "Transformer Inference Arithmetic" offered as a text substitute for the second half. The Friday session is marked build and skim, with the note that the value is producing your own figure rather than re-reading the explanation.

Measurement lands in week 3 and load testing in week 5, before the knobs

The second ordering decision is that instrumentation comes before optimization. This is the part of the plan most likely to be skipped by an impatient reader, and skipping it removes the ability to verify anything that follows. The README states it without hedging: nothing after them is verifiable without them, and a public benchmark is mostly a credible measurement harness with a model attached.

By the end, the stated inventory includes Grafana dashboards showing TTFT, inter-token latency, throughput, queue depth, and cost per request, plus a load test harness that reproducibly drives 1000 or more concurrent requests. The tooling listed for installation is short: vllm, guidellm, and sglang via pip, then Docker for the Prometheus and Grafana stack, and kind or a small managed cluster for week 8.

The third decision is a deliberate refusal to build. Paged attention and continuous batching are not build exercises, because vLLM implements both and chunked prefill is the default scheduling strategy in vLLM and SGLang. The README's position is that reimplementing them teaches less than reading the block manager and the scheduler until you can explain from the code why memory waste drops under 4 percent and why the GPU stops idling between requests. That is a defensible call, and it is also the point where the roadmap stops being hands-on for a stretch. Weeks 2 and 4 are largely code reading.

The router is where the dollar figure enters the code

The fourth decision pairs the cost/latency/quality router with unit economics in the same week. The README's justification is that a router picking a backend by cost, latency, and quality is the one component that forces a dollar figure and a latency budget into code, and studying economics alongside it turns the reading into a routing policy rather than a blog post you agreed with. The deliverable is a router with per-request token budgeting.

This is the strongest design choice in the roadmap. Cost per request appears on the Grafana dashboard and again as a routing input, which means the number has to survive contact with an actual decision. Most curricula treat inference economics as a reading list item. Here it is a function signature.

The fifth decision is that edge deployment comes last and is optional. ONNX Runtime, TensorRT-LLM, and WebLLM are described as client-side and embedded runtimes that share almost no operational surface with the datacenter stack the other nine weeks build. Placing them at the end, rather than interleaving them, keeps the main artifact coherent.

Getting it running, and the GPU bill nobody itemizes up front

The installation is three pip commands and a Docker dependency. There is no repository code to clone for the service itself, no Makefile, no pinned requirements file in the material supplied. The roadmap is a syllabus with links, and the artifact is something you write.

GPU access is where the plan gets expensive and where the README is unusually specific. A 7-8B model needs roughly one 24GB GPU, rented by the half hour and shut down between sessions. RunPod is listed for per-second billing and quick pods, Modal for serverless and benchmark sweep scripts, Lambda and vast.ai for cheap on-demand and marketplace capacity, Colab for pure-Python sessions with no serving.

The cost-reduction guidance is the useful part: all of week 1, most of week 3, the week 6 lecture days, and all of week 9 reading need no GPU at all, and rentals should be batched around build days. One H100 is needed for exactly two sessions, the 1000-concurrent test in week 5 and disaggregation in week 8 if run for real. That is two H100 sessions against 25 hours of total material, which is a manageable ratio if you plan the rentals. It is not manageable if you leave an instance running between Tuesday and Thursday.

Where the plan is thin, and the case against it

The roadmap assumes a 7-8B model fits one 24GB card, and the later weeks assume that same model survives FP16, FP8, INT4, speculative decoding, and KV eviction variants for benchmarking. If your target model is larger, or if your quantization toolchain does not support the architecture you picked, the GPU budget in the README stops applying and the plan needs rework. The material does not address that case.

The buffer days are described as catch-up time, not new material, and the README is explicit that missing a Tuesday should not collapse the plan. Fifty sessions at 30 minutes is 25 hours, which is a real commitment spread over ten weeks, and the plan has no mechanism for someone who falls two weeks behind other than compressing the skim sessions. It warns against compressing the build sessions, which is correct and also leaves little slack.

The deeper limitation is that this is a reading order, not a codebase. There are no releases, no pinned dependency versions in the repository itself, and no reference implementation to diff against. If you want to see how someone else wired the router or the load harness, you will not find it here. Compare that with vLLM's own documentation and examples, which ship runnable server launch commands and a supported-models matrix. The difference in approach is that vLLM's material answers "how do I run this" and this roadmap answers "in what order should I learn the things that let me run this well." They are not substitutes. A reader who wants the first will find the roadmap frustratingly indirect for the first two weeks.

Licence, maintenance, and what the repository actually carries

The repository is Apache-2.0, which permits commercial use, modification, and redistribution with the usual notice and patent grant terms. Since the material is prose and links rather than code, the practical implication is limited: you can reuse the curriculum inside a team or a course. This is not legal advice, and if you plan to redistribute a modified version, read the licence text rather than this paragraph.

Maintenance is the open question. There are no releases, and the repository's last push is dated 2026-08-14. The content depends on external links, and several of them are to specific lecture recordings and blog posts that can move or disappear. The pip packages it tells you to install, vllm, sglang, and guidellm, ship on their own schedules, and the README does not pin versions. A pinned-versions benchmark writeup is one of the stated end deliverables, which means you produce the version pinning yourself. That is the correct design for a curriculum that cannot track upstream releases, but it does mean the roadmap will drift out of date in proportion to how fast the serving stack changes, and nothing in the repository signals when that has happened.

Editorial conclusion

Adopt this if you already write Python and understand transformer architecture but have never stood up a serving stack, and you can rent GPU time in half-hour blocks. Do not adopt it if you need Kubernetes or CUDA depth, or if you want a reference implementation rather than a syllabus: the repository ships a reading order, not code. Before starting, verify that your chosen 7-8B model fits a 24GB card at the quantization levels the later weeks assume, and price the two H100 sessions, because those are the only line items the README cannot absorb into a per-second rental.

Official sources

1. Issues
2. License: Apache-2.0
3. patchy631/time-to-first-token on GitHub
4. README