Dream-RSI: a research preview of exploration-layer self-improvement, with the codebase still pending

The bottleneck Dream-RSI targets: exploration policy, not the coding agent

Most self-improvement systems improve the thing that writes code. Dream-RSI goes after the layer that decides where to search. The README frames the problem directly: as self-improvement targets get harder, discovery stretches over thousands of proposal and evaluation cycles, and a poor exploration strategy spends computation on directions that go nowhere. The stated dilemma is that fixed strategies cannot adapt as the search space scales, while tuning the policy online means searching a meta-space under feedback that is both delayed and expensive, since judging one exploration policy means watching how it shapes an entire discovery run.

The intended audience is narrow. This is for people building or studying automated discovery loops, where each evaluation is a real code execution with a real cost, and where the orchestration layer (branching, parallel exploration, stopping) can be separated from the model doing the work. If your pipeline has no expensive search phase, or if a single evaluation is cheap enough to rerun freely, the problem Dream-RSI solves does not exist for you.

History as a replay simulator: the mechanism behind the dreaming loop

The core claim is that a finished discovery process is already a dataset. According to the README, a completed run records a structured tree of past exploration decisions and their realized code-execution outcomes. A different policy can traverse that same tree differently: other subsets of recorded branches, other orders, other parallel groupings, other stopping decisions. Because the outcomes are already saved, evaluating the alternative requires only reading past records, with no rerun of the discovery agent or the evaluator. The README draws the analogy to model-based RL and world models, calling the history a world the agent dreams in.

The loop has three stated stages. The current policy drives online discovery and logs its traces. Those recorded trees are converted into a reusable simulator pool. Candidate policies are then evaluated and refined by dreaming over that pool, which returns immediate, low-cost off-policy feedback, and the improved policy is redeployed online, expanding the pool further. The README describes this as an orchestration layer that makes exploration explicit and programmable while leaving the underlying coding agent unchanged.

That last clause is the part worth pausing on. The method is a meta-controller, not a new model. It also means the simulator is only as rich as the trees you have already collected: a policy that would explore a region your history never touched cannot be scored by replay, and the README does not describe how the pool is extended beyond what online runs happen to produce.

What the repository actually contains today, and what the release plan still lists as pending

There is no install section to write, because there is nothing to install. A note near the top of the README states that code is being prepared for release and points to the release plan. That table marks the paper PDF and the project page as available, arXiv posting as in progress, and discovered programs, the full codebase and reproduction scripts as being prepared.

The repository layout matches that: the top-level entries are .gitignore, CITATION.cff, README.md, assets/ and papers/. The paper sits at papers/Dream-RSI.pdf, and the README points readers to dream-rsi.com for the method, results and an interactive walkthrough. So the honest first step is reading, not running. If you want to cite it, the README supplies a BibTeX entry:

@article{zheng2026dreamrsi,
  title   = {Dream-RSI: Recursive Self-Improvement through Evolving Worlds},
  author  = {Zheng, Tong and Wu, Xidong and Zhang, Zheng and He, Zhankui and
             Zhang, Chaoyi and Coleman, Benjamin and Wei, Ruoqiao and Bai, Di and
             Liu, Haolin and Liu, Rui and Wang, Xue and Zhuan, Yue and
             Kang, Wang-Cheng and Xiang, Renkai and Huang, Heng and
             Cheng, Xinwu and Guo, Yunsong},
  journal = {arXiv preprint arXiv:2609.14858},
  year    = {2026}
}

The README does not document an installation path, a Python package, a command line entry point, environment variables or configuration keys. Anything you see claiming otherwise is not from this repository.

The reported results, and why the baseline choice matters more than the multipliers

The README's stats graphic reports several comparisons, and the caption is unusually careful about what each number is measured against. Algorithm engineering shows 1.22x faster downstream runtime, 1.74x less discovery compute and 162x fewer calls than SimpleTES. Mathematical optimization reaches 2 of 3 tasks at or above the selected baseline. GPU kernel engineering reports 4 of 4 kernels improved, 2.09x higher performance at an equal budget and 2.43x fewer generations at equal performance. The caption also states zero gradient steps on the coding agent, and that comparisons are against Recursive Fixed Exploration unless a published system is named, with algorithm engineering run on Gemini-3.1-Pro.

Two things deserve attention. First, the headline comparison against SimpleTES is the only one naming a published system; the rest are internal, against a fixed-exploration variant of the same method. Beating your own fixed baseline is a weaker claim than beating an outside system, and the README does not present an external comparison for the math or kernel tasks. Second, 2 of 3 tasks at or above baseline is a partial result, not a sweep. The README reports it that way rather than rounding up, which is to its credit, but it also means the method is not uniformly better across the three domains it lists.

Where the replay-simulator assumption breaks

The method's efficiency argument rests entirely on outcomes being reusable. That holds when the environment is effectively stationary: the same evaluator, the same runtime, the same task definition, so a recorded outcome still means what it meant when it was produced. It does not hold when the evaluator changes, when the task is redefined mid-run, or when outcomes depend on state that the tree does not capture.

There is a subtler failure mode. Replaying a tree can only score policies that stay inside branches already taken. A policy whose value comes from trying something no prior run attempted is invisible to the simulator, and the README does not describe a mechanism for detecting that blind spot or for deciding when to spend real compute on genuinely new directions rather than dreaming over old ones. Systems that learn a world model from logged interaction share this exposure, and it is the standard reason such methods need periodic real-environment checks. Dream-RSI's own loop provides that check through redeployment, but the README does not state how often redeployment should happen or what triggers it.

If your discovery process is short, or if a single evaluation costs seconds, the simulator layer is overhead with no payoff. Rerunning the search is simpler and exact.

How it differs from a world-model RL setup and from plain search

The nearest comparison is a learned world model in model-based RL. There, a network is trained to predict transitions, and the model is approximate by construction, which is why rollouts drift and need correction. Dream-RSI's simulator is not learned at all: it is a replay of recorded outcomes, so it is exact for any trajectory the tree contains and silent about everything else. That is a real trade: no training cost and no prediction error, but coverage limited to the realized search space.

The other comparison is to the fixed exploration strategy the README uses as its baseline. Recursive Fixed Exploration keeps one policy throughout and spends its budget on search. Dream-RSI spends part of the budget on choosing the policy. The reported 1.74x reduction in discovery compute and 162x fewer calls versus SimpleTES are the README's argument that the overhead pays for itself, at least in the settings measured. Whether it pays off in a shorter run, where there is little history to replay, is not something the README addresses.

Licence, maintenance and the cost of tracking a moving release

The repository has no licence file listed among its top-level entries, and the README does not state a licence. Treat the contents as all rights reserved until a licence appears, and do not assume the paper's availability implies a permissive code licence. The CITATION.cff file exists for academic attribution, which is a separate matter from redistribution rights.

The last push was on 2026-09-16, so the repository is current. That currency is mostly README and asset churn ahead of a code drop, not a maintained library with a release cadence. There are no releases retrieved. The upgrade cost question is therefore premature: what you would be tracking is a release plan, and the README's own table is the thing to watch, since it is where a change in the status of the full codebase, reproduction scripts or discovered programs would appear. Until those rows change, planning an integration means planning against a paper.

Editorial conclusion

Dream-RSI is worth reading now if you work on meta-search, agent orchestration or evaluation reuse, because the replay-simulator idea is the transferable part and the PDF is already in the repository. It is not adoptable as a library today: the release plan lists the full codebase, reproduction scripts and discovered programs as still being prepared, so there is nothing to pip install or clone into a working pipeline. Before you plan around it, check the release plan table in README.md for a status change on the full codebase, and read the paper's task definitions to see whether your own discovery logs have the branch structure the simulator needs.

Frequently asked questions

Official sources

1. Issues
2. README
3. zhengkid/Dream-RSI on GitHub