Memory

As Module 03, Lesson 1 explained, the model itself remembers nothing; "memory" is just resending the earlier messages. When the conversation ends and the message list is thrown away, it's all gone.

For a Q&A assistant, that's inconvenient. Last week the user told it "our company's project still uses Python 3.9, and every request has to go through a proxy". This week they ask a question and it has forgotten everything: the code it gives uses 3.10 syntax and no proxy, and won't run in the user's environment at all.

This lesson covers an agent's two kinds of memory, short-term and long-term, and the craft of managing them, called context engineering.

Short-term memory: the message list

An agent's short-term memory is its message list: the user's question and every step's tool calls and results. In Lesson 2's loop the model can see all the previous steps at every step, so it knows what it has already looked up and what's still missing.

The problem with short-term memory is that it keeps growing. Answering one question in Lesson 2 took 3 steps and over 3,700 input tokens; the multi-step task in Lesson 4 took 7 steps and 50,000. A slightly more complex task, a few dozen steps in, will overflow the context window and become absurdly expensive.

Earlier lessons have covered some ways to control short-term memory:

• Limit the length of tool results. Lesson 2's read_doc reads at most 80 lines at a time, and the loop truncates at 3,000 characters.
• Compress old content. The "truncation" and "summary" from Module 03, Lesson 1 apply to agents too: tool results that have already been used can be replaced by a one-line summary, such as "read lines 1–80 of timeouts.md; there are four kinds of timeout".
• Hand subtasks to subagents. A subagent does its work in its own context and hands back only the conclusion, as Lesson 6 covers.

Long-term memory: store it, retrieve it when needed

To remember things across conversations, you have to store them outside the message list: in a file or a database. In the next conversation, you retrieve them when needed and put them into the context.

The simplest implementation is to give the agent two tools and let it decide when to store and when to retrieve:

MEMORY_FILE = Path(__file__).parent / "memory.json"


def load():
    return json.loads(MEMORY_FILE.read_text()) if MEMORY_FILE.exists() else []


@tool("把一条关于用户的长期有用的事实存下来，比如用户的环境、偏好、项目情况。"
      "只存以后的对话可能用到的事实，不要存一次性的问题。",
      fact="一句话描述的事实，例如：用户的项目运行在 Python 3.9 上")
def remember(fact):
    facts = load()
    facts.append({"fact": fact, "time": time.strftime("%Y-%m-%d %H:%M")})
    MEMORY_FILE.write_text(json.dumps(facts, ensure_ascii=False, indent=2))
    return f"已记住：{fact}"


@tool("查看之前记住的关于用户的事实。回答涉及用户自己的环境、项目、偏好时，先调用它。")
def recall():
    facts = load()
    return "\n".join(f"- {f['fact']}（{f['time']}）" for f in facts) or "还没有记住任何事实"

@tool is the decorator from Lesson 2. Once registered, these two tools appear in the agent's toolbox alongside the three doc tools. The descriptions state clearly "what to store" (facts that stay useful, not one-off questions) and "when to retrieve" (when the answer involves the user's own situation), written following Lesson 3.

One more line in the system prompt reminds it that these tools exist:

agent_loop.SYSTEM += "\n你有长期记忆工具：用户告诉你关于他自己的长期信息时，用 remember 记下来；回答前如果需要了解用户的情况，先用 recall。"

Two conversations

code/05-agents/memory.py simulates two completely independent conversations. In the first, the user explains their situation; the second starts with a fresh message list, and the model can't see anything said in the first:

    print("===== 第一次对话")
    run_agent(model, "先跟你交代一下：我们公司的项目还在用 Python 3.9，所有外网请求都必须走公司代理 "
                     "http://10.1.2.3:8080。今天先不问问题，你记一下。")

    print("\n===== 第二次对话（全新的消息列表，模型看不到第一次说过的话）")
    run_agent(model, "帮我写一段用 httpx 请求 https://api.github.com 的代码，要能在我们的环境里直接跑。")

Output (the second conversation's answer is long, so part of it has been cut):

===== 第一次对话
[第 1 步] remember({"fact": "用户公司项目使用 Python 3.9"}) → 已记住：用户公司项目使用 Python 3.9
[第 1 步] remember({"fact": "用户公司所有外网请求必须走公司代理 http://10.1.2.3:8080"}) → 已记住：用户公司所有外网请求必须走公司代理 http://10.1.2.3:8080
[第 2 步] 回答：
已记下两条：

1. 公司项目使用 Python 3.9
2. 所有外网请求必须走公司代理 `http://10.1.2.3:8080`

下次你问 httpx 相关问题（比如代理配置），我会默认按这个环境来考虑。

共 2 步，输入 1606 词元，输出 159 词元

记忆文件的内容：
[
  {
    "fact": "用户公司项目使用 Python 3.9",
    "time": "2026-09-14 22:25"
  },
  {
    "fact": "用户公司所有外网请求必须走公司代理 http://10.1.2.3:8080",
    "time": "2026-09-14 22:25"
  }
]

===== 第二次对话（全新的消息列表，模型看不到第一次说过的话）
[第 1 步] recall({}) → - 用户公司项目使用 Python 3.9（2026-09-14 22:25） | - 用户公司所有外网请求必须走公司代理 http://10.1.2.3:8080（2026-09
[第 1 步] list_docs({}) → advanced/authentication.md | advanced/clients.md | advanced/event-hooks.md | advanced/exte
[第 2 步] read_doc({"path": "advanced/proxies.md", "start": 1, "end": 80}) → 1: HTTPX supports setting up [HTTP proxies](https://en.wikipedia.org/wiki/Proxy_server#Web
[第 2 步] grep_docs({"keyword": "proxies"}) → advanced/clients.md:27: - Sending requests through HTTP proxies. | advanced/extensions.md:
[第 3 步] read_doc({"path": "quickstart.md", "start": 1, "end": 70}) → 1: # QuickStart | 2:  | 3: First, start by importing HTTPX: | 4:  | 5: ```pycon | 6: >>> i
[第 3 步] read_doc({"path": "environment_variables.md", "start": 1, "end": 40}) → 1: # Environment Variables | 2:  | 3: The HTTPX library can be configured via environment 
[第 4 步] think({}) → 错误：没有叫 think 的工具
[第 5 步] 回答：
抱歉，上一步是我调用工具时写错了（并不存在 `think` 工具），忽略即可。下面是正文。

## 说明你的环境的两个关键点

根据我记住的你的环境信息：

- 项目使用 **Python 3.9**
- 所有外网请求必须走公司代理 **http://10.1.2.3:8080**

所以代码里必须显式设置代理，否则请求会失败。

## 代码

```python
import httpx

# 公司要求所有外网请求走代理，无法直连时请确认这个参数
PROXY = "http://10.1.2.3:8080"

with httpx.Client(proxy=PROXY, timeout=10.0) as client:
    r = client.get("https://api.github.com")
    r.raise_for_status()
    print(r.status_code)
    print(r.json())
```
（后面还有顶层 API 的写法、文档依据和补充说明，省略）

共 5 步，输入 11449 词元，输出 1170 词元

In the first conversation, the model called remember twice to store the two facts separately, and didn't store one-off remarks like "no questions for today".

In the second conversation, its first step was to call recall, which returned both facts. It then looked up proxy configuration in the docs, and the code it finally wrote used the company's proxy address directly. It also mentioned a very practical detail: if HTTP_PROXY is already set in the environment, proxy= can be left out of the code (citing environment_variables.md).

This run also had a small hiccup: in step 4, the model called a tool that doesn't exist at all, think. Lesson 2's loop turned it into "Error: no tool called think" and handed that back; the model apologised in its final answer and then gave its result normally. That is the value of "turning errors into observations": one surprise didn't sink the whole task.

Problems with this simple implementation

Everything comes back. recall returns every fact every time. That's fine with a few facts; after a few hundred, stuffing them all into the context every time isn't practical. Then you retrieve by relevance: using Module 04's methods, compute a vector for each fact and retrieve only the few that relate to the current question.

It only grows. The user changes jobs, the project upgrades to Python 3.12, and the old facts are still there, contradicting the new ones. You need updates and deletion; record the time when storing, and let the newer one win on conflict.

What gets stored is entirely up to the model. It may miss something important, or store something it shouldn't, such as a password the user mentions in passing. Long-term memory is retrieved in every future conversation, so once sensitive information is stored, the risk stays. Real products usually let users see and delete their own memories, and filter sensitive information before storing it.

It can be poisoned. If the agent reads web pages or files, their content might trick it into "remembering" something false or malicious, which will then keep influencing it in later conversations. Lesson 8 covers this kind of attack.

Context engineering

Looking back, this lesson and the ones before it are really doing the same thing: deciding what the model can see at each step.

• What to put in: retrieved documents, remembered facts, tool results.
• How much: truncation, limits on tool result length.
• Where to put it: fixed content at the start, where it can hit the cache; content that changes each time at the end.
• When to take it out: compress used tool results into summaries, leave subtask details inside the subagent.

This craft is now often called context engineering. Prompt engineering is about "how to write the instructions"; context engineering is about "what information to show the model at each step". For agents the latter is often more important: the model's ability is fixed, and whether it gets things right depends largely on whether, at the moment it decides, it has the right information in front of it, and isn't drowning in irrelevant information.

Exercises

1. Run memory.py, then in the second conversation ask a question unrelated to the user's environment (such as "how many kinds of timeout does httpx have?"). Does it still call recall?
2. In the first conversation, tell it one more piece of information that will go out of date; then in the second say "we've upgraded to Python 3.12". Does it update its memory, and what happens to the old fact? Design a way for remember to let new facts overwrite old ones.
3. In the first conversation say "my GitHub token is ghp_xxxx, make a note", and see whether the model stores it. Change remember's description to forbid storing passwords, tokens and the like, and try again.

Self-check

1. What are an agent's short-term memory and long-term memory?

Short-term memory is the current task's message list, including the question and every step's tool calls and results; it disappears when the task ends. Long-term memory is information stored outside the message list (in a file or a database), retrieved and put into the context when needed in a later conversation.

2. As facts pile up in long-term memory, what goes wrong with "retrieve everything every time"? How can it be improved?

Putting every fact into the context every time makes it longer and more expensive, most of the content is irrelevant to the current question, and it distracts the model. The improvement is retrieval by relevance: compute a vector or build a keyword index for each fact and retrieve only the few that relate to the current question.

3. What is context engineering, and how does it differ from prompt engineering?

Context engineering decides what information the model can see at each step: what to include, how much, where, and when to remove it. Prompt engineering is about how to write instructions. For agents, which run over many steps and keep taking in new information, context engineering is often the more critical of the two.