Project: a Q&A assistant that reads the source

RepoBot v2 has one limit it can't get around: it only reads the docs. "What is httpx's default maximum number of redirects?" isn't in the docs, so all it can say is "no relevant information found in the documentation". Yet the answer is right there in the httpx source, one grep away.

This lesson turns RepoBot into an agent that decides for itself whether to search the docs or dig through the source.

What done looks like

• Ask "what is httpx's default maximum number of redirects?" and the answer is 20, citing [源码 httpx/_config.py:248] (源码 = source).
• For questions the docs answer, it prefers the docs, citing [文档 file name] (文档 = docs).
• For questions unrelated to httpx, it declines directly without calling any tool.
• Passing a path such as ../ that escapes the source directory to read_source gets refused.
• All 8 questions in eval_agent.py (5 of them answerable only from the source) are answered correctly.

Structure

The code is in projects/repobot/v3/:

tools.py          四个工具：search_docs、read_doc、grep_source、read_source
agent.py          智能体循环（05 模块第 2 课的写法）
repobot.py        命令行对话程序
retrieval.py      检索，沿用 v2
llm.py            模型客户端、计费、重试，沿用 v2
eval_agent.py     8 道题的评估

Four tools

v2's retrieval is wrapped as a tool, plus three new tools:

The descriptions follow Lesson 3's method, each stating clearly when the tool should be used. Take grep_source as an example:

@tool("在 httpx 的 Python 源码里搜索一段文本或正则表达式，返回文件路径、行号和那一行，最多 30 条。"
      "文档里找不到答案时使用，比如某个参数的默认值、某个异常在什么情况下抛出、某个函数的内部逻辑。",
      pattern="要搜索的文本或正则表达式，例如 DEFAULT_MAX_REDIRECTS 或 def raise_for_status")
def grep_source(pattern):
    try:
        regex = re.compile(pattern)
    except re.error:
        regex = re.compile(re.escape(pattern))
    ……
    if not hits:
        return f"源码里没有找到 {pattern}，换个写法再试，比如只搜函数名或常量名"

Two details: the regex the model sends may be malformed, so if it fails to compile, the tool falls back to a plain text search instead of raising an error; and when nothing is found, the message tells the model what it can do next.

The source is cloned automatically on the first run:

def ensure_source():
    """第一次运行时把 httpx 的源码克隆下来。"""
    if not (SOURCE_DIR / "httpx").is_dir():
        print(f"第一次运行，正在从 {SOURCE_REPO} 下载 httpx 源码……", flush=True)
        SOURCE_DIR.parent.mkdir(parents=True, exist_ok=True)
        subprocess.run(["git", "clone", "--depth", "1", "--quiet", SOURCE_REPO, str(SOURCE_DIR)], check=True)

--depth 1 downloads only the latest version, not the whole history, which is much faster.

The two file-reading tools share a check function that ensures the model can only read files in the designated directories:

def inside(base, path):
    """把相对路径转成绝对路径，并确认它没有跑出 base 目录。跑出去了就返回 None。"""
    target = (base / path).resolve()
    return target if base in target.parents and target.is_file() else None

As Lesson 8 explained, every tool an agent has is a potential entry point for abuse. RepoBot only needs to read, so all four tools are read-only, and what they can read is limited to the docs and source directories.

The prompt: docs first, then source

SYSTEM = """你是 RepoBot，Python HTTP 客户端库 httpx 的答疑助手。你可以查 httpx 的官方文档和源码。

做法：
- 先用 search_docs 查文档。文档里有答案，就根据文档回答。
- 文档里没有答案（比如默认值、内部逻辑、某个异常什么时候抛出），再用 grep_source 和 read_source 查源码。
- 回答里注明依据：文档写成 [文档 文件名]，源码写成 [源码 文件路径:行号]。
- 只根据查到的内容回答。查了还是找不到，就如实说没有找到，不要猜。
- 和 httpx 无关的问题，直接礼貌地说明你只负责 httpx，不要调用任何工具。
- 用中文回答，简洁，代码保持原样。"""

Why docs first? Docs are written for users and say "how it should be used"; source is implementation detail and may contain internal behaviour that isn't promised to anyone. If the docs can answer, don't dig it out of the source. Also, one docs search is usually enough, while digging through source often takes several searches and several reads, which costs more.

The "answer in Chinese" rule is the lesson from Lesson 4: after reading lots of English material, an agent's answer sometimes starts in English.

Running it

cd projects/repobot/v3
pip install -r requirements.txt
export HF_ENDPOINT=https://hf-mirror.com
python repobot.py

For a question v2 can't answer, this time let's ask a trickier one: "If I write only httpx.Limits(max_connections=200), what is max_keepalive_connections?" The docs only say "(Defaults 20)", which easily leads people to think the answer is 20.

你：只写 httpx.Limits(max_connections=200)，max_keepalive_connections 是多少？
  [1] search_docs({"query": "Limits max_connections max_keepalive_connections default"}) → [1] 文档 advanced/resource-limits.md
  [1] grep_source({"pattern": "max_keepalive_connections"}) → httpx/_config.py:167: * **max_keepalive_connections** - Allow the conn
  [2] read_source({"path": "httpx/_config.py", "start": 160, "end": 200}) → 160:     """
RepoBot：**是 `None`**（即不限制 keep-alive 连接数）。

原因：`httpx.Limits(...)` 构造时 `max_keepalive_connections` 的参数默认值是 `None` [源码 httpx/_config.py:177]，你不传它就保持 `None` [源码 httpx/_config.py:181]。

注意别把两个"默认"混淆：

- `Limits` 构造函数的默认值是 `None`
- 而 **客户端在没有传入 `limits` 时**使用的 `DEFAULT_LIMITS = Limits(max_connections=100, max_keepalive_connections=20)` [源码 httpx/_config.py:247]，文档里说的 "(Defaults 20)" 指的是这个 [文档 advanced/resource-limits.md]
（后面的代码示例省略）
[3 次模型调用，3 次工具调用，本轮 0.00080 美元，累计 0.00080 美元]

It searched the docs and the source at the same time, read lines 160 to 200 of _config.py in step 2, and then distinguished two easily confused "defaults". I checked this part of the source in Module 03, Lesson 5: in the Limits constructor max_keepalive_connections defaults to None, and DEFAULT_LIMITS is on line 247. The answer is entirely correct.

Evaluation: 8 questions, three runs

eval_agent.py has 8 questions: 3 whose answers are in the docs and 5 whose answers are only in the source. Each question has a regular expression, and an answer that matches counts as correct.

The first run scored 8/8. But I read every answer, and on the Limits question it had said "it's 20; the effective default comes from the module-level constant", which is wrong. It merely mentioned None in passing in its explanation, and my regex r"None" marked it "correct".

This is the problem from Module 01, Lesson 6: scoring scripts can be wrong too. So I added a "must not appear" regex to each question, to catch answers that "mention the keyword but reach the wrong conclusion":

QUESTIONS = [
    # (问题, 必须出现, 不能出现, 答案在哪)
    ……
    ("只写 httpx.Limits(max_connections=200)，max_keepalive_connections 是多少？", r"None", r"是\s*\**\s*`?20", "源码"),
]

After the fix, I ran it twice more:

########## v3 评估第 1 次
答案在文档里的题：3/3 答对
答案在源码里的题：5/5 答对
平均每题 2.6 次模型调用，2.5 次工具调用，共 0.0067 美元
……
########## v3 评估第 2 次
答案在文档里的题：3/3 答对
答案在源码里的题：5/5 答对
平均每题 2.5 次模型调用，2.5 次工具调用，共 0.0063 美元

Both times, the Limits question was answered correctly ("max_keepalive_connections will be None"). Across three runs, one run got one question wrong.

This shows two things. First, an agent's answers differ every time; the same question can be right this time and wrong next time, so one evaluation run doesn't tell you much, and you need several. Second, the stricter the automatic scoring rules, the more real problems they catch, but they can also penalise correct answers. Both are what the next module tackles systematically.

An engineering trap: threads and local models

The evaluation script runs 4 questions at once in a thread pool. Once I started two evaluation programs at the same time, and one of them hung for over ten minutes without printing a single result, while CPU usage sat near 400%.

The cause was the local embedding model. Every thread was calling it, and PyTorch starts several threads of its own when computing. Several Python threads times PyTorch's threads, times two processes, gave far more threads than CPU cores, all fighting each other so none could make progress.

The fix is a lock, so only one thread calls the local model at a time:

_MODEL_LOCK = threading.Lock()  # 同一时间只让一个线程调用本地的嵌入模型和重排模型
……
    def vector_search(self, query, k):
        with _MODEL_LOCK:
            q = self.embedder.encode(["query: " + query], normalize_embeddings=True)[0]

Computing the vector for one question takes only a few milliseconds, so queueing barely affects speed. Waiting on the model API can still happen in parallel.

Compared with v2

v3 is more capable, and also more expensive and less predictable. In our evaluation it cost only about 30% more than v2, because most questions are settled in two or three steps.

Questions this project should answer

• Why this design? Answers the docs don't give are in the source, and when to dig through the source, and where, can't be hard-coded in advance, so we use an agent. All tools are read-only and limited to two directories.
• Where will it fail? The same question may get a different answer each time; the agent may find a related but wrong piece of code in the source and draw the wrong conclusion from it; when the question involves call relationships across several files, it may not read everything it needs.
• How is it evaluated? eval_agent.py, run several times after each change. The automatic scoring is still crude; the next module improves it.
• What do you look at when something goes wrong? Every tool call is printed, so you can see what it searched for and which lines it read. The next module records these as logs.
• Can it be cheaper? It can answer with v2's fixed workflow first and start the agent only when the answer is "not found in the documentation" (the strategy from Lesson 1).
• Does it really need an agent? For questions whose answers are in the source, yes. For most docs questions, not really, which is exactly the basis for the previous point's optimisation.

Exercises

1. Ask v3 a question that needs tracing across files, such as "in which function does httpx.get actually send the network request?", and see how far it gets and whether its conclusion is right.
2. Implement the mixed strategy from Lesson 1: run v2's workflow first, and start v3's agent only when the answer contains "not found in the documentation". Compare the total cost over the 8 evaluation questions.
3. Add a parameter to eval_agent.py that runs each question 3 times and counts how often each is answered correctly. Which questions are "reliably right", and which are "sometimes right, sometimes wrong"?

Self-check

1. Why does RepoBot search the docs first and the source second, rather than going straight to the source?

The docs describe the usage promised to users, while the source contains lots of internal implementation detail that isn't necessarily stable public behaviour. Also, one docs search is usually enough, while digging through the source takes several searches and reads, which is more expensive and slower. Only when the docs don't have the answer does it need to look in the source.

2. The evaluation script showed 8/8. Why still read every answer?

Automatic scoring can be wrong. In this lesson's first run, one question reached the wrong conclusion but happened to mention the keyword in its explanation, so it was marked correct. Only by reading the answers yourself do you know whether the scoring rules are reliable, and can improve them, for example by adding "must not appear" conditions.

3. Why do several threads calling a local embedding model at once slow down to a near standstill? How do you fix it?

PyTorch starts several threads of its own for each computation. With several Python threads calling the model at once, the thread count multiplies, far exceeding the number of CPU cores, and they fight over resources. The fix is a lock so only one thread calls the local model at a time. Each computation is fast, so queueing barely affects speed.