How to design tools

How useful an agent is depends mostly on its tools. All the model can see is each tool's name, description and parameter definitions, never your code. If the description is vague, the model can only guess: what does this tool do? When should I use it? What goes in the parameters?

This lesson starts with an experiment that measures the difference between good and bad descriptions, then covers specifically how to write them.

Experiment: vague descriptions versus clear ones

The same three functions: search the docs, read a doc file, look up a version number on PyPI. Two sets of descriptions.

The vague set: names are generic verbs, and descriptions are two or three words:

VAGUE = [
    fn("search", "搜索", q="内容"),
    fn("read", "读取", x="要读的东西"),
    fn("lookup", "查找信息", name="名字"),
]

The clear set: names say what they operate on, and descriptions state what the tool does, when to use it and how to fill in the parameters:

CLEAR = [
    fn("search_docs", "在 httpx 官方文档里按英文关键词全文搜索，返回匹配的文件名和行号。"
       "用户问 httpx 某个功能怎么用、某个参数是什么意思时，先用它。",
       keyword="英文关键词，例如 timeout、proxy、follow_redirects"),
    fn("read_doc", "读取 httpx 文档里某个文件的内容。通常在 search_docs 找到文件名之后使用。",
       path="文档文件路径，例如 advanced/timeouts.md"),
    fn("get_pypi_info", "查询某个 Python 包在 PyPI 上的最新版本号和发布信息。只在用户问版本号、是否已发布新版本时使用。",
       package="PyPI 上的包名，例如 httpx"),
]

(fn is a small function that builds a tool description; full code in code/05-agents/tool_design.py.)

Then 10 questions, each labelled with which tool should be called first. For "hello", the right answer is to call no tool at all. Each question is asked 3 times with each set of descriptions, looking only at which tool the model picks for its first step, without actually running it:

QUESTIONS = [
    ("httpx 怎么设置代理？", "search_docs"),
    ("httpx 最新版本是多少？", "get_pypi_info"),
    ("帮我看看 advanced/ssl.md 里写了什么", "read_doc"),
    ("follow_redirects 参数是干什么的？", "search_docs"),
    ("requests 现在出到哪个版本了？", "get_pypi_info"),
    ("httpx 怎么上传文件？", "search_docs"),
    ("你好", None),
    ("把 quickstart.md 的内容给我看一下", "read_doc"),
    ("httpx 有没有发布 1.0 正式版？", "get_pypi_info"),
    ("httpx 的 event hooks 怎么用？", "search_docs"),
]

Results:

含糊的工具：14/30 次选对
    httpx 怎么设置代理？  应该用 search_docs，实际 {'None': 2, 'search_docs': 1}
    httpx 最新版本是多少？  应该用 get_pypi_info，实际 {'search_docs': 3}
    帮我看看 advanced/ssl.md 里写了什么  应该用 read_doc，实际 {'read_doc': 2, 'get_pypi_info': 1}
    follow_redirects 参数是干什么的？  应该用 search_docs，实际 {'search_docs': 1, 'None': 1, 'get_pypi_info': 1}
    requests 现在出到哪个版本了？  应该用 get_pypi_info，实际 {'get_pypi_info': 1, 'search_docs': 2}
    httpx 怎么上传文件？  应该用 search_docs，实际 {'None': 3}
    httpx 有没有发布 1.0 正式版？  应该用 get_pypi_info，实际 {'search_docs': 3}
清楚的工具：30/30 次选对

The same model, with only the descriptions changed, went from 47% correct to 100%.

What's wrong with the vague descriptions

The tool's scope is unclear. "Search" where? Web pages, docs or code? The model doesn't know. So for "what's the latest httpx version" it chose search all 3 times, because "search" sounds the most general.

It's unclear when to use it. For "how do I upload a file with httpx", it called no tool all 3 times and answered straight from memory. It didn't know search could help it find a more reliable answer, so it had no reason to use it.

The name doesn't match the function. lookup actually checks PyPI versions, but the description "find information" can't be told apart from "read" and "search", so the model could only pick at random. For "what does the follow_redirects parameter do", its 3 attempts gave 3 different choices.

The clear descriptions spell all of this out: the search covers "the official httpx documentation", "use it first when the user asks how to use some httpx feature"; the version tool is "only for when the user asks about version numbers". The model doesn't need to guess.

Names

• Say what it operates on. search_docs is better than search, get_pypi_info better than lookup. Once there are many tools, generic names easily collide.
• Start with a verb, and be consistent. get_, search_, read_, create_: pick one set of rules and stick to it.
• Don't abbreviate. You know what gpi means; the model doesn't.

Descriptions

A good description answers three questions:

1. What does it do? "Full-text search of the official httpx docs by English keyword; returns matching file names and line numbers."
2. When should it be used? "Use it first when the user asks how to use some httpx feature."
3. When should it not be used? "Only use when the user asks about version numbers or whether a new version has been released."

The third point matters especially when tools are easily confused. You can also describe how tools work together, as in read_doc's "usually used after search_docs has found the file name", so the model knows to search first and read after.

Parameters

• Describe every parameter, ideally with an example. "English keyword, e.g. timeout, proxy, follow_redirects". The example shows the model the format, and also hints that "the docs are in English, so search in English".
• The fewer parameters the better. With seven or eight parameters, the model easily leaves one out or fills one wrongly. Give defaults wherever possible.
• Restrict values with enums. When a parameter can take only a few fixed values, list them with enum in the schema (as in Module 02, Lesson 4).
• Clear names and types. Parameter names like x, q or name are worse than path, keyword or package.

Return values

A tool's return value goes into the context as-is, and every later step pays for it. So:

• Return only useful information. Last lesson's get_pypi_info picked just four fields (version number, summary, Python version requirement), rather than stuffing in the tens of KB of raw JSON PyPI returns.
• Limit the length. Last lesson's grep_docs returns at most 20 matches, read_doc at most 80 lines at a time, and the loop truncates at 3,000 characters on top of that.
• Make the next step easy for the model. grep_docs returns "file name:line number:content", so the model can call read_doc directly with the file name and line number.
• Say clearly when the result is empty. Return "no matches for pool timeout", not an empty string. An empty string confuses the model: is the tool broken, or is there really nothing?

Error messages

Error messages are written for the model and should help it correct itself:

错误：没有这个文件 advanced/timeout.md，请先用 list_docs 查看有哪些文件

This one sentence makes three things clear: what went wrong, which parameter was wrong, and what to do next. Compare Python's default FileNotFoundError: [Errno 2] No such file or directory: the model can understand it too, but doesn't know which tool to call to find the right file name.

When there are many tools

The more tools, the harder it is for the model to choose correctly, and the longer the descriptions sent with every request. Some lessons from experience:

• Merge tools with similar functions. If even you can't say what the difference between search_docs and search_api_reference is, merge them into one with a parameter to tell them apart.
• Group tools by scenario. Give each task only the tools relevant to it; Lesson 6's multi-agent setup uses this idea.
• Read the traces. A tool that's often misused is a tool whose description needs work.

Exercises

1. In the clear set in tool_design.py, delete "use it first when the user asks how to use some httpx feature" from search_docs's description and rerun. Does the result for "how do I upload a file with httpx" change?
2. Add a function to both sets: list_docs, which lists all doc files. In the vague set call it list with the description "list"; in the clear set write it following this lesson. Add two questions to test it.
3. Take a function you've written before, write a tool description for it following this lesson, and let the model call it.

Self-check

1. Which questions should a good tool description answer?

What it does; when it should be used; when it should not be used (especially when it's easily confused with another tool). You can also say how it works with other tools, such as "usually used after search_docs".

2. Why should tool return values be as short as possible?

The return value goes into the message list as-is, and every later step of the agent carries it into the model call and pays for it. Overlong return values also make it hard for the model to find what matters, and can even fill the context window. Return only useful fields and set a length limit.

3. What makes a good tool error message?

It's written for the model, saying what went wrong, where, and what to do next. For example, "no file named X; use list_docs first to see which files exist". That way the model can correct itself instead of repeating the same mistake.