Project: the first version of the Q&A assistant

Everything in this module is easy on its own. This lesson assembles it into a complete small program: RepoBot v1, an assistant that answers httpx questions on the command line. It is the starting point of the project that runs through all of Part 1, and the next few modules will improve it one version at a time.

What done looks like

Set the goal before you start. When you finish this lesson, all of the following should hold:

• Run python repobot.py and you can hold a continuous conversation with it; it remembers what was said in earlier turns.
• Answers stream onto the screen token by token.
• After each turn it shows input and output tokens, cache hits, the cost of the turn and the running total.
• Ask something unrelated to httpx and it politely declines.
• If the network drops or the server errors, the program doesn't crash; it asks you to ask again.
• You can say which kinds of questions it gets wrong, and why.

The last point matters most. v1 is deliberately imperfect. Only once you see its problems clearly do you know what v2 needs to fix.

Structure

The full code is in projects/repobot/v1/repobot.py, a little over a hundred lines in four parts:

repobot.py
  SYSTEM        system 提示词：身份、范围、规则
  cost_usd      根据 usage 算钱（01 模块第 4 课）
  open_stream   发起流式请求，连接阶段出错自动重试（本模块第 2、4 课）
  answer        流式打印回答，拼出完整文本，检查是否被截断（本模块第 2 课）
  main          多轮对话循环，维护历史、打印花费（本模块第 1 课）

Each part was covered in an earlier lesson, so below we only discuss the new questions that come up when putting them together.

The system prompt

SYSTEM = """你是 RepoBot，Python HTTP 客户端库 httpx 的答疑助手。

- 只回答和 httpx 有关的问题，包括它的用法、原理、报错排查，以及和 requests 等库的比较。
- 和 httpx 无关的问题，礼貌地说明你只负责 httpx，不要回答。
- 回答要简洁，能用代码说明的就给代码。
- 不确定的地方要明确说"我不确定"，不要编造版本号、参数名或者更新日志。"""

Four rules, each aimed at a specific problem (the approach from Module 02, Lesson 1): the first sets the scope; the second stops it from becoming a general assistant that chats about anything, which is both product positioning and cost control; the third controls the length and form of answers; the fourth targets hallucination. Whether the fourth one actually works, the test below will tell us.

Combining streaming with retries

The call_llm from Lesson 4 was written for non-streaming calls. Streaming adds a complication: an error can happen when half the answer has already been printed.

RepoBot handles this by splitting a streaming call into two phases:

def open_stream(messages, max_attempts=4):
    """发起流式请求。连接阶段出错会自动重试；开始输出之后再出错，就不重试了。"""
    for attempt in range(1, max_attempts + 1):
        try:
            return client.chat.completions.create(
                model=MODEL,
                messages=messages,
                stream=True,
                stream_options={"include_usage": True},
                max_tokens=4000,
                extra_body={"thinking": {"type": "enabled" if THINKING else "disabled"}},
            )
        except RETRYABLE as e:
            if attempt == max_attempts:
                raise
            wait = 2 ** (attempt - 1) + random.random()
            print(f"\n[{type(e).__name__}，{wait:.1f} 秒后重试]", file=sys.stderr)
            time.sleep(wait)

If the error happens while connecting (rate limit, server error, connection failure), the user hasn't seen anything yet, so it is safe to retry. Once output has started, an error midway is not retried, because a regenerated answer won't match the first half the user has already read. main catches that error, tells the user "this turn was discarded, you can ask again", and leaves the turn out of the history:

        try:
            text, usage = answer(messages)
        except openai.APIError as e:
            print(f"\n[出错了：{type(e).__name__}，这一轮作废，可以再问一次]")
            continue

The answer function prints each piece of text while also saving it to a list, then joins them into the full answer to store in the history. It also records finish_reason; if that is length, it tells the user the answer was cut off.

The history uses the simplest truncation: keep only the last 20 messages. In a Q&A setting, users rarely ask a dozen questions in a row and still need to refer back to the very first one.

Running it

cd projects/repobot/v1
pip install -r requirements.txt
python repobot.py

I prepared 6 questions to test it. Five are about httpx and have definite answers, which I looked up in the httpx source beforehand; the last one has nothing to do with httpx. I fed them in all at once through a pipe:

printf '%s\n' "httpx 默认会自动跟随重定向吗？" "那默认最多跟随几次重定向？" \
  "httpx 的 Client 默认连接池最多允许多少个连接？keep-alive 连接最多多少个？" \
  "httpx 0.27.2 这个版本修复了什么问题？" "怎么给 httpx 设置 10 秒的超时？给一行代码。" \
  "今天北京天气怎么样？" | python repobot.py

Below is the complete output from my run, unedited except that I removed a few overly long code examples. Your answers will be worded differently, and may even differ in which ones are right.

你：httpx 默认会自动跟随重定向吗？
RepoBot：是的，httpx 默认会自动跟随重定向（`follow_redirects=True` 是默认行为）。

注意和 requests 的区别：**requests 默认跟随，httpx 也是默认跟随**，但 httpx 早期版本默认是 `False`，0.20 之后改成了 `True`。如果你用的是很老的版本，行为可能不同。
（代码示例省略）
具体默认值建议以你所用版本的文档为准，我不确定所有版本都是 `True`。
[输入 118（缓存命中 0），输出 219，本轮 0.00030 美元，累计 0.00030 美元]

你：那默认最多跟随几次重定向？
RepoBot：默认最多跟随 **20 次**。
（代码示例省略）
补充一点：httpx 的 `TooManyRedirects` 继承自 `RequestError`，可以直接用 `httpx.RequestError` 捕获。

这个 `20` 是 httpx 的默认值，和 requests 的默认 `30` 不同。我不确定该默认值在所有历史版本里都是 20，建议以你所用版本的文档为准。
[输入 349（缓存命中 128），输出 221，本轮 0.00033 美元，累计 0.00063 美元]

你：httpx 的 Client 默认连接池最多允许多少个连接？keep-alive 连接最多多少个？
RepoBot：`httpx.Client` 的默认连接池上限是 **100 个连接**，其中默认 **20 个是 keep-alive 连接**。
（代码示例省略）
- `max_keepalive_connections`：池中保留的、可复用的空闲连接数上限，默认为 `max_connections` 的 20%（即 100 的 20% = 20）。如果显式只设 `max_connections`，这个值会按比例走。
- `keepalive_expiry`：空闲 keep-alive 连接的存活时间，默认 5 秒。
（后面还有两条补充说明，省略）
[输入 598（缓存命中 384），输出 405，本轮 0.00055 美元，累计 0.00118 美元]

你：httpx 0.27.2 这个版本修复了什么问题？
RepoBot：我不确定 httpx 0.27.2 具体修复了哪些问题，不想凭印象编造更新日志。

建议你直接看官方来源核实：

- 发布记录：https://github.com/encode/httpx/releases
- 更新日志文件：https://github.com/encode/httpx/blob/master/CHANGELOG.md

在 Releases 页面里找到 `0.27.2` 那个 tag，就能看到该版本的具体改动。如果你把 changelog 内容贴过来，我可以帮你解读这些改动对你用法的实际影响。
[输入 1022（缓存命中 768），输出 119，本轮 0.00022 美元，累计 0.00141 美元]

你：怎么给 httpx 设置 10 秒的超时？给一行代码。
RepoBot：```python
r = httpx.get("https://example.com", timeout=10.0)
```
（后面还有两种写法，省略）
[输入 1162（缓存命中 1024），输出 81，本轮 0.00014 美元，累计 0.00155 美元]

你：今天北京天气怎么样？
RepoBot：抱歉，我只负责 httpx 相关的问题，天气这类问题我答不了。如果你有 httpx 用法、报错或配置方面的问题，我很乐意帮忙。
[输入 1252（缓存命中 1024），输出 35，本轮 0.00012 美元，累计 0.00167 美元]

Checking each answer

I checked the answers sentence by sentence against the httpx source (httpx/_client.py, httpx/_config.py, httpx/_exceptions.py).

Question 1: wrong. httpx's Client has follow_redirects: bool = False, so it does not follow redirects by default, which is one of the important differences from requests. The model not only got it wrong but invented a version history ("changed to True after 0.20") to back up the wrong answer. It added "I'm not sure" at the end, but everything before that sounded completely confident, and most users would believe it.

Question 2: right. The source has DEFAULT_MAX_REDIRECTS = 20. TooManyRedirects does inherit from RequestError. (I didn't check the requests source for its default of 30, so that part doesn't count.)

Question 3: the numbers are right, the explanation is made up. The source has DEFAULT_LIMITS = Limits(max_connections=100, max_keepalive_connections=20) and keepalive_expiry defaults to 5 seconds, so all three numbers are correct. But "max_keepalive_connections defaults to 20% of max_connections, so setting only max_connections scales it proportionally" is invented: in the Limits class the default for max_keepalive_connections is None, and there is no proportional calculation anywhere. This kind of "correct numbers with a made-up mechanism" is the hardest to catch.

Question 4: nothing invented. In Module 01, Lesson 2, the same question got a nonexistent security vulnerability, CVE number and all. This time it said "I'm not sure" and pointed to where to look it up. The difference is the line in the system prompt: "where you're unsure, say clearly 'I'm not sure'; don't make up version numbers, parameter names or changelogs." The rule helps, but questions 1 and 3 show it can't stop all fabrication: the model first has to "realise" it is unsure before the rule takes effect.

Question 5: right.

Question 6: declined gracefully.

Now the bill: 6 turns cost $0.00167 in total. Cache hits grow every turn (0, 128, 384, 768, 1024), because the system prompt and the earlier history form a fixed prefix, and the caching from Module 01, Lesson 4 kicks in automatically.

Where v1 falls short

Of 6 questions: 2 fully correct, 1 declined gracefully, 1 honest "I don't know", 1 with correct numbers but a made-up explanation, and 1 flat wrong with an invented justification.

There is only one root cause: it can only answer from memory. The model's training data contains a lot about both httpx and requests, and their APIs are very similar, so the memories get mixed up. Question 1 most likely took requests' behaviour and attributed it to httpx.

Tweaking the prompt does little for this. The real fix is to have it look up the official httpx documentation before answering, answer from the documentation, and tell the user which page the answer came from. That is RAG, the next module.

Questions this project should answer

After each version, check your design against these questions:

• Why this design? Command line + streaming + truncated history is the simplest form that works. Get something usable first, then add features.
• Where will it fail? Obscure defaults, version differences, and things that look like requests but behave differently are easy to get wrong, and it is confident when it's wrong.
• How do you know whether it's good? So far, only 6 hand-checked questions. Module 06 grows them into an evaluation set that runs automatically.
• What do you look at when something goes wrong? Right now, only the output on screen. Module 06 adds logging.
• Can it be cheaper? It's already cheap. Flash without thinking costs less than $0.001 per turn.
• Does it really need an agent? No. v1 is just one call plus conversation history. Only in Module 05, when it has to decide for itself whether to search the docs or read the source, do we consider an agent.

Exercises

1. Turn on thinking mode with --think and ask the 6 questions again. Does it get questions 1 and 3 right? What does it cost now?
2. Delete the "where you're unsure, say clearly…" rule from the system prompt, ask question 4 a few more times, and see whether the model goes back to inventing changelogs.
3. Write 5 httpx questions of your own, preferably ones whose answers you can confirm in the docs or source, test v1 on them and count how many it gets right. Keep these 5 questions; you'll test v2 on them in the next module.

Self-check

1. Why doesn't RepoBot retry an error that happens after output has started?

The user has already seen part of the answer. A retry generates a new answer from scratch, which probably won't match the first half the user saw, and the two pieces together would be confusing. So it only retries while connecting (before the user has seen anything); once output has started, an error just asks the user to ask again.

2. The system prompt says "if unsure, say you're unsure". Why did RepoBot still make things up on question 1?

The rule only takes effect when the model "realises" it is unsure. On question 1 the model wrongly believed it knew the answer, so it gave the wrong answer with confidence. Prompts can reduce fabrication but can't cure it. The cure is to have the model answer from real material, which is RAG.

3. Why do RepoBot's cache hits keep growing each turn?

Every turn's request starts with the same system prompt and the earlier conversation history. What was the previous request becomes part of this turn's prefix, so the server can reuse the results it already computed, and the matching part keeps getting longer.