Module 03 · Building with LLMs

Building apps on the API

From a single call to a real program: multi-turn chat, streaming, tool calling, error retries and cost tracking, ending with the first version of the running project, RepoBot.

Lessons

01
Multi-turn chat: the model remembers nothing
Write a command-line chat program and see for yourself that the model has no memory, and that "memory" is just resending the message history every time. Then compare two ways of handling a history that gets too long, truncation and summarization.
35 min · Beginner
02
Streaming output
Show the answer as it's being generated. Measure time to first character with and without streaming, handle usage and thinking content while streaming, then push the model's output to a browser in real time with FastAPI.
35 min · Intermediate
03
Tool calling: letting the model act
A model can't look up live data or carry out any action by itself. Give it a real tool that checks a package's latest version on PyPI, and walk through the whole tool-calling flow, including parallel calls, error handling and what to watch in thinking mode.
45 min · Intermediate
04
Errors, retries, rate limits and cost
Reproduce the most common API errors, see which retries the openai SDK already does for you by default, then write a call function with timeouts, exponential backoff, a concurrency limit and cost logging that you can drop straight into a project.
40 min · Intermediate
05
Project: the first version of the Q&A assistant
Assemble multi-turn conversation, streaming, retries and cost tracking into RepoBot v1, a Q&A assistant for httpx. Test it on 6 questions with known answers and see what it gets right and what it makes up.
60 minutes · Intermediate

In the first two modules every example was "call once, print the result". A real application faces more: users ask several questions in a row; answers are long and users won't wait; you need live data; networks drop, services rate-limit you, and the bill has to add up.

This module solves these problems one by one and then assembles the solutions into the first version of the running project, RepoBot: an httpx Q&A assistant that chats continuously in the terminal, streams its output, retries automatically and shows the cost of each turn.

Why this order

Lesson 1 first works out what a "conversation" really is at the API level; every lesson after it builds on the message list. Lesson 2's streaming improves the user experience and stands fairly apart. Lesson 3's tool calling is the most important and the most complex, and it's the foundation of module 05's agents. Lesson 4 handles what every one of these features runs into in the real world: errors and money. Lesson 5 assembles the first four.

You're done when

You can write a command-line chat program, and truncate or compress the history when it gets too long.
You can stream an answer so it shows as it's generated, and know how to forward it to a browser.
You can write a complete tool-calling loop from scratch: the tool descriptions, execution, passing results back, handling parallel calls and tools that fail.
You can write a call function with timeouts, exponential-backoff retries, a concurrency limit and cost tracking.
RepoBot v1 runs on your computer, and you can say which kinds of question it gets wrong and why.

Code for this module

Code and program output are shown exactly as they ran, so comments and printed output are in Chinese.