recursive-llm keeps your document in a Python REPL instead of your prompt

The problem is prompt cost, not context length alone

Long-context models let you paste a large document into a prompt, but you pay for that paste on every call, and a task that needs several passes pays several times. recursive-llm takes the opposite route. The README states the design directly: instead of placing the full source in model prompts, the library keeps it in a restricted Python REPL where the model can search, compute, partition, and send only selected sections to language-model calls. The audience is narrow and identifiable. It is for engineers whose source text is too large or too expensive to place in a normal prompt and whose task involves search, filtering, exact counting, or local Python computation. It is explicitly not for the case where the context fits comfortably in the model window and the task is small. The README says so in the same section that recommends the library, which is a useful signal about how the authors see the trade-off.

The root model never sees the whole document

The execution diagram in the README is the clearest description of the mechanism: query to root model to Python REPL over full context, then selected chunks or child calls, then answer, with shared budgets and trajectory alongside. The root model receives the query and instructions, while the source is exposed inside the REPL as the variable named context. REPL state persists across iterations, so the model can inspect a small region, run a computation, then inspect another region without re-sending what it already read. From inside that loop it has four moves: local search or computation, a plain LM call, a child RLM call when the configured depth permits it, or returning an answer. That depth limit matters. A child RLM is not free, and the README's own scale check notes that the 1M-character runs used local REPL computation without child RLMs, so the deepest configuration is not the one the authors exercised at the largest size.

Installation is from source, and the entry-point guard is mandatory

The package is not yet published to PyPI, so there is no pip install recursive-llm. The README gives two routes: clone the repository and run pip install -e ., or install the current GitHub version with pip install "recursive-llm @ git+https://github.com/grishahq/recursive-llm.git". Python 3.9 or higher is required, plus an API key for your provider or a local model setup such as Ollama or llama.cpp. A minimal script constructs RLM(model="gpt-5-mini") and calls complete_result with a query and a context string read from a UTF-8 file, then prints result.answer and result.stats. The constraint that will bite people first is stated plainly: RLM uses a spawned worker process for isolated REPL execution, so executable Python scripts must use the standard if __name__ == "__main__": guard. This is a Python multiprocessing requirement on spawn-based platforms, not a style preference. Code that runs the model at import time will fail. Model routing goes through LiteLLM, which is what makes OpenAI, Anthropic, DeepSeek and local providers reachable through the same model string, and what makes cost estimation possible at all.

Budgets and statistics cover the whole recursion tree

The README lists tree-wide limits for calls, tokens, estimated cost, elapsed time, and local execution. That is the part that distinguishes this from a thin wrapper: a recursive run can spawn child runs, and a per-call limit would not stop a tree from growing. RLM.stats aggregates model calls across the complete recursion tree and breaks them into llm_calls, root_calls, recursive_calls and leaf_calls, alongside prompt_tokens, completion_tokens, cached_tokens and estimated_cost_usd, with a by_model breakdown. Two details in the README are worth reading twice. Each root completion receives fresh statistics, so the numbers describe one recursion tree rather than lifetime usage. And estimated_cost_usd is None when LiteLLM has no pricing metadata for any contributing model, which means a provider that LiteLLM does not price will silently remove your cost ceiling. If cost control is why you are here, check that field before trusting the budget.

What the benchmark table does and does not say

The measured results come from one generated corpus seed and three live runs per configuration on a 100k-character aggregation benchmark. GPT-5 mini went from 0/3 exact passes in direct mode to 3/3 under RLM, with mean model tokens falling from 37,928 to 8,224 and mean estimated cost from $0.0088788 to $0.0048132. DeepSeek V4 Flash went from 0/3 to 2/3, with tokens falling from 39,364 to 15,209. Three runs per configuration is a small sample, and the README says as much: these measurements are an engineering check, not a paper reproduction or a universal quality claim. It also states that direct completion was faster and cheaper on short tasks, and that a separate 1M-character scale check passed 6/6 exact-graded runs using local REPL computation without child RLMs. None of these numbers should be read as a general quality claim. They describe an aggregation task where exact counting is the point, which is precisely the shape of problem a REPL-based approach should win.

Where it is the wrong tool

The README names the failure mode itself: semantic synthesis across long narrative text remains difficult and should be evaluated on your own data before production use. That is a real boundary, not a hedge. A REPL is good at slicing, counting and filtering. Judging which of forty passages matters most, or writing a coherent summary that depends on relationships spread across a whole book, does not reduce to a Python expression, and the model still has to decide what to send upward. There is a second boundary around execution. The restricted worker runs in a spawned process with hard timeouts and optional POSIX resource limits, which the README lists as a feature. It also means the local execution budget is a real constraint you have to set, not a formality. And the Python 3.9 to 3.12 badge is a floor and a ceiling: anyone on 3.13 is outside the tested range. A third boundary is operational. There is no hosted service and no published package, so upgrades arrive as Git commits and releases you pull yourself, and the CI badge plus the v0.3.1 release titled CI Tooling Fix suggest the tooling around the library has needed attention recently.

The alternative is direct completion, and the difference is where the text lives

The comparison the README makes is against direct completion: one prompt containing the full source, one model call, one answer. The difference is architectural rather than a matter of tuning. In direct mode the entire document occupies the prompt on every call, so cost scales with document size times number of passes, and the model's attention is spread across everything at once. In RLM mode the document sits in a Python process, the model sees only what it selects, and cost scales with what it chooses to read plus the calls it makes. Direct completion wins on latency and on short inputs, which the README concedes. The README also points to a separate document-format evaluation comparing the two approaches on a SHA-pinned TXT book, PDF, CSV and HTML documentation page, with two reliability and performance changes evaluated one at a time. The project also credits the original implementation at alexzhang13/rlm and the underlying paper, so this is one of at least two codebases implementing the same idea, and the README frames its own contribution as provider portability, tree-wide budgets, structured failures, reproducible benchmarks and complete run trajectories rather than the algorithm itself.

Licence, maintenance and what to check before you commit

The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are included. That is the standard reading of the text, not legal advice; if you are embedding this in a product, have someone qualified review the LICENSE file and the licences of the dependencies you pull in alongside it, since LiteLLM and your provider SDKs carry their own terms. Maintenance cost is dominated by two things. First, the library is not on PyPI, so your upgrade path is a Git URL or a local editable install, and pinning matters more than usual. Second, model strings and pricing metadata come from LiteLLM, so a provider rename or a pricing gap shows up as a None cost estimate rather than an error. The repository has a SECURITY.md and a DOCUMENT_EVALUATION.md, and the release cadence shows v0.3.0 in late July 2026 and v0.4.0 in early September 2026, titled Reliable Long Document Processing. Before adopting, run the quickstart against one of your own documents, confirm the entry-point guard is in place, and check the by_model block in rlm.stats to see which models actually received calls and whether cost was estimated or None.

Editorial conclusion

Adopt it if your task is aggregation, exact counting, filtering or search over source text that will not fit in a prompt, and you want one budget covering the whole recursion tree. Do not adopt it for short prompts (the README states direct completion was faster and cheaper there) or for semantic synthesis across long narrative text, which the README lists as difficult. Before committing, verify two things yourself: that a fresh clone plus pip install -e . works on your Python version, and that rlm.stats.estimated_cost_usd is not None for your chosen provider, since it is None when LiteLLM has no pricing metadata.

Official sources

1. grishahq/recursive-llm on GitHub
2. Issues
3. License: MIT
4. README
5. Releases