Module 02 · Building with LLMs

Prompting

Comparison experiments that make clear how a prompt is structured, few-shot examples, thinking before answering and JSON output, ending with a small tool for testing prompts in bulk.

Lessons

01
What a good prompt looks like
Run the same messy request for help once with a dashed-off prompt and once with a clearly structured one, and compare the results section by section. What role, task, background, requirements and output format each contribute.
35 min · Beginner
02
Giving examples: few-shot prompting
Sort user messages into four categories. With no examples, 18 are right; with 4 examples, 20 are. How to choose the examples, how many to include, and the side effects they bring.
30 min · Beginner
03
Making the model think before it answers
Six tricky questions, five runs each. Answering directly gets 21 right, writing out the reasoning first gets 29, and thinking mode gets them all. Why thinking before answering helps, and what it costs.
35 min · Beginner
04
Getting JSON out of the model
When a program has to process the model's answer, it needs reliably formatted JSON. JSON mode, validating with Pydantic and having the model fix its own failures, and getting schema-conformant structured output from strict-mode tool calls.
40 min · Beginner
05
Prompts need tests too
Put prompts in files, prepare 30 test cases, run each 3 times, and compare two versions of a prompt with data. You'll also see that test results need checking themselves: sometimes the mistake isn't the model's but the label's.
40 min · Beginner

Every lesson in this module is a set of comparison experiments: the same task, written two ways, with the results side by side. You'll see a prompt dashed off in one line make the model write the user's guess down as fact, four examples get every classification right, and one instruction to "write out the reasoning first" cut the mistakes on a trick question from 9 to 1.

There's no "universal template" here. Whether a prompt is good depends on the results it gets on your task, so the last lesson teaches you how to test prompts in bulk.

Why this order

Lesson 1 covers the parts a prompt is made of, which is the foundation. Lessons 2 and 3 each cover one of the most-used techniques built on it: giving examples, and making the model think before it answers. Lesson 4 turns to processing by programs, covering how to make the output reliable JSON, which applications and agents later can't do without. Lesson 5 puts everything before it into a testing framework: before and after you change a prompt, let the data speak.

You're done when

Given a one-line prompt, you can find what's wrong with it and rewrite it with a role, task, requirements, format and delimiters.
You can pick suitable few-shot examples for a classification task and say why you picked them.
You can judge whether a task needs thinking mode, and back your judgement with data.
You can write code with "JSON mode + Pydantic validation + retry on failure" that reliably extracts structured data from text.
After changing a prompt, you can use a set of test cases to show where the new version is better than the old and where it's worse.

Code for this module

Code and program output are shown exactly as they ran, so comments and printed output are in Chinese.