# MemAgent, a memory agent that turns long-context reading into an RL-trained workflow

> MemAgent is a ByteDance and Tsinghua SIA research framework that adds a recurrent memory mechanism to an LLM and trains the whole agent workflow with reinforcement learning from verifiable rewards, so that a model trained on 32K tokens extrapolates to 3.5M token question answering. The repository ships a quickstart script, trained 7B and 14B weights, and a multi-conversation RL recipe built on an extended DAPO algorithm.

**BytedTsinghua-SIA/MemAgent** — A MemAgent framework that can be extrapolated to 3.5M, along with a training framework for RL training of any agent workflow.

- Repository: https://github.com/BytedTsinghua-SIA/MemAgent
- Stars: 1,113 · Forks: 74
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/bytedtsinghua-sia-memagent

## A recurrent memory inside the context window

MemAgent is not a longer context window. The stated mechanism is a memory architecture that enables arbitrarily long input processing within fixed context windows, described as breaking the usual context length limitation, and the project attributes to it linear time complexity, meaning resources scale linearly with text length rather than quadratically. The model architecture itself is not altered in the usual sense: the framing is end to end reinforcement learning over long-context tasks without changing the underlying model. The reported numbers are specific. The model extrapolates from an 8K context trained on 32K text to a 3.5M token QA task with performance loss under 5 percent, and reaches over 95 percent accuracy on the 512K RULER test. The released RL-MemAgent models come in two sizes, with the 14B model degrading under 5.5 percent on 3.5M token tasks and the 7B model showing an 11 percent decline at the longest contexts. The repository also points to a paper, a blog and a Hugging Face dataset for HotpotQA.

## Multi-conversation RLVR, DAPO extended to agent workflows

The training story is a separate contribution from the memory design. MemAgent uses Reinforcement Learning from Verifiable Rewards, extending the DAPO algorithm to support end to end optimization of agent workflows built on multi-turn context-independent conversations. That phrase carries the weight: the unit being optimized is a sequence of turns where each turn is a separate conversation, not a single long prompt, and the reward is computed from a checker rather than a human or a preference model. The repository layout reflects that this is a research codebase with the training stack vendored in. The top level includes `verl/`, `recipe/`, `recurrent/`, `serve/`, `taskutils/`, `examples/`, `tests/` and `scripts/`, and the `examples/` directory carries subfolders for data preprocessing, generation, a grpo_trainer, a ppo_trainer, ray setups, reinforce_plus_plus_trainer, remax_trainer, rloo_trainer, sft, slurm, split_placement and tuning. The `run_memory_14B.sh`, `run_memory_7B.sh`, `run_memory_7B_async.sh` and `run_tool_gsm8k_async.sh` scripts at the root are the entry points for those runs.

## Serving the 14B model and pointing quickstart at it

The quickstart script is the shortest path to a working system, and the README offers two ways in. For a local deployment you start a vLLM server on the released 14B weights, then run the script against that same model name. The tensor parallel size of 2 in the serve command is the only tuning knob shown, and it implies a multi-GPU machine rather than a laptop. For an online service the script reads a model endpoint and an API key from environment variables, with the normal case being a plain endpoint URL and the Azure OpenAI case needing the deployment path appended to it. The script name is the same in both cases, only the endpoint configuration differs.

```bash
vllm serve BytedTsinghua-SIA/RL-MemoryAgent-14B --tensor_parallel_size 2
```

The server has to be up before the script runs, and the second call names the same model so the script and the server agree on what is being served.

```bash
python quickstart.py --model BytedTsinghua-SIA/RL-MemoryAgent-14B
```

## Pointing the same script at a hosted model

The online path is worth reading because it shows the script is not tied to a local server. The endpoint is exported as `URL` and the credential as `API_KEY`, and the model argument then names the model you want to drive. The Azure example shows the URL format explicitly, with the deployment segment and the model name appended to the endpoint, which is the part people get wrong when they paste a plain endpoint into a tool that expects a deployment path. Once the variables are set, the invocation is the same `python quickstart.py --model` line used for the local case.

```bash
export URL=
export API_KEY=
python quickstart.py --model gpt-4o-2024-11-20
```

The development dependency list at the root of the repository is long and pins its own versions, including `flash-attn`, `liger-kernel`, `hydra-core`, `peft`, `tensordict<=0.6.2` and `ray[default]`, with a commented `vllm==0.6.3.post1` line and `wandb` for experiment tracking, so training a variant means installing the whole development set rather than the inference subset.

## The verifier gap that makes reproduced scores look wrong

This is the sharpest practical note in the documentation, and any reader who reproduces the numbers needs it. The README says that in a reproduction the validation score during training is not equal to the final score, roughly 50 percent against 80 percent, and states that this behaviour is expected. The reason is that training used a stricter verifier than testing did. The training verifier, in `verl/utils/reward_score/hotpotqa.py`, requires the answer to sit inside `\boxed{}` with exact case matching and no additional characters. The testing verifier, in `taskutils/memory_eval/utils/__init__.py`, ignores articles such as a and the, and ignores case differences and punctuation. The stricter rule was inherited from earlier math oriented RL work, while the relaxed rule follows the practice of long-context projects such as Ruler and Qwen-Long. In other words the two numbers are not measuring the same thing, and a model that looks bad during training is not necessarily bad at the task.

## Reproducing the evaluation, and what it costs

The testing path is documented as a sequence of preparation steps, and it is heavy. A QA dataset is downloaded with a shell script in `taskutils/memory_data`, then the dataset itself is pulled with `hfd.sh` using aria2c and a `DATAROOT` environment variable pointing at the result. Models come from Hugging Face, but the Qwen2.5-Instruct series has to be downloaded by hand and its `config.json` edited to activate YaRN, following the instructions in the Qwen2.5-Instruct repository, and the three sizes 7B, 14B and 32B are then moved under a `MODELROOT` path with `-128K` suffixes. The evaluation itself runs `python run.py` from `taskutils/memory_eval`, and the README warns that a full run takes a few days and that you may want to select which tests and models to run. That script uses all available GPUs to serve models, and for several nodes you can build a ray cluster and use `SERVE_PORT` and `DASH_PORT` to set the cluster ports.

## A research repository, not a packaged product

The maintenance picture is the main caveat for anyone planning to depend on this. The repository is not archived, but the last push was on 2026-05-12, more than five months before now, and there are no GitHub releases, so there is no tag to pin and no changelog of published versions. The only dated artifacts are the news items in the README, a quickstart script added in 2025/07 and the RL-MemAgent-14B and RL-MemAgent-7B models released in 2025/06. The licence is Apache-2.0, with the text in `LICENSE` and a `Notice.txt` at the root, which is a permissive licence for research and commercial reuse alike. The practical reading is that this is a paper repository. The model weights on Hugging Face are the durable artifact, and the code is best treated as a reference implementation you fork rather than a dependency you upgrade.

## Conclusion

Adopt MemAgent if your workload is a single long document or a long corpus read as question answering, and if you have multi-GPU hardware and appetite for the training side, because the interesting part of this repository is the recipe rather than the inference path. Skip it if you need a general purpose long context assistant, since the published gains are measured on QA and RULER style benchmarks and the memory agent is a different object from a plain model. Verify three things first: the verifier behaviour, because the training verifier demands an exact `oxed{}` answer and the testing verifier is deliberately more lenient, which is why a reproduced validation score near 50 percent can still correspond to a final score near 80 percent; the `vllm==0.8.2` requirement if you intend to reproduce rather than only serve; and the fact that the repository has no tagged releases, so the model weights and the code are the only artifacts you can pin.

## FAQ

### What is MemAgent and what problem does it solve?

It is a long-context processing framework that adds a recurrent memory mechanism to a language model and optimizes long-context tasks end to end with reinforcement learning, without changing the model architecture. The goal is to process arbitrarily long input within a fixed context window with linear scaling of resources.

### How do I run MemAgent with the released models?

Serve the weights with vLLM and then run the quickstart script against the same model name, for example `vllm serve BytedTsinghua-SIA/RL-MemoryAgent-14B --tensor_parallel_size 2` followed by `python quickstart.py --model BytedTsinghua-SIA/RL-MemoryAgent-14B`. The weights for the 7B and 14B variants are on Hugging Face.

### Why does my reproduced validation score differ so much from the final score?

The documentation says this is expected. Training used a stricter verifier that requires the answer inside `\boxed{}` with exact case matching and no extra characters, while testing uses a more lenient verifier that ignores articles such as a and the, case differences and punctuation.

### How is MemAgent trained?

With reinforcement learning from verifiable rewards, extending the DAPO algorithm to optimize agent workflows made of multi-turn context-independent conversations. The repository vendors the training stack under `verl/`, with run scripts for the 7B and 14B models and separate recipe directories under `examples/`.

### Can MemAgent be used with an online model service instead of a local server?

Yes. The quickstart script supports online services by reading the endpoint from a `URL` environment variable and the credential from `API_KEY`. For Azure OpenAI the endpoint needs the deployment path and model name appended, in the form `https://{endpoint}/openai/deployments/gpt-4o-2024-11-20`.

## Sources

- [BytedTsinghua-SIA/MemAgent on GitHub](https://github.com/BytedTsinghua-SIA/MemAgent)
- [Issues](https://github.com/BytedTsinghua-SIA/MemAgent/issues)
- [License: Apache-2.0](https://github.com/BytedTsinghua-SIA/MemAgent/blob/main/LICENSE)
- [README](https://github.com/BytedTsinghua-SIA/MemAgent/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/bytedtsinghua-sia-memagent