EvoMap/AutoResearch: a stateful agent pipeline that turns a research idea into paper-ready evidence

The failure mode AutoResearch was built around

Ask a single language model to propose a research direction, implement it, run it, and judge the result, and you get a loop with no external check. The README names the problem directly: research agents can invent missing details when evidence is thin and repeatedly validate their own output. AutoResearch is aimed at researchers and small labs who want an agent to carry a hypothesis through planning, coding, execution and evaluation, but who do not trust a self-graded result. The stated goal is paper-ready evidence, meaning plans, code, run logs, metrics, failure causes, critic reports and blind reviews, all written to disk so a human can inspect or take over. It is not a paper writer. It is an evidence generator with a paper-shaped output.

How the pipeline keeps state and where independence is enforced

The workflow is stateful and recoverable, and that is the architectural claim worth examining. Plans, code, queues, logs and conclusions are persisted, so a run interrupted mid-experiment can resume rather than restart. Idea generation works by intersecting two inputs: online research signals (recent papers, community discussions, open-source trends, aggregated and deduplicated) and a local knowledge base in knowledge_base/ that holds your own research experience, constraints and known failure patterns. The README states the workflow never rewrites that knowledge base automatically, which matters because it means the agent cannot quietly drift your domain constraints. Independence is enforced at the model level: stages that require independent opinions count distinct underlying model identities, and Idea Generation is expected to use at least three distinct models. A single endpoint can expose several models, so the constraint is on identity rather than on vendor. The pilot-before-scaling step is the other cost control: feasibility is tested cheaply before a full run is committed.

Installing AutoResearch and running the first preflight check

The README targets a Linux or SSH machine with Git, Python 3.10+ and python3-venv. Clone and run the bringup script:

git clone https://github.com/EvoMap/AutoResearch.git
cd AutoResearch
bash scripts/bringup.sh

bringup.sh creates .venv, installs the Python dependencies, runs the baseline tests and a secret scan, and checks the current model configuration. According to the README it does not contact model services and incurs no API charges. Expect a final BLOCKED result or a nonzero exit on the first run, before credentials exist; that is documented behaviour, not a broken install. What you should verify is that Python setup, dependency installation and tests all succeeded.

Next, create the two local configuration files without clobbering anything:

test -f .env || cp .env.example .env
test -f config/providers.local.json || \
  cp config/providers.example.json config/providers.local.json

.env holds real API URLs, keys and proxies and should not be committed. config/providers.local.json declares endpoints, model aliases, and which model fills each role. The .env.example file lists the variables you may need to fill, including ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY, GEMINI_BASE_URL, GEMINI_API_KEY, OPENAI_BASE_URL, OPENAI_API_KEY, GPT_CRITIC_BASE_URL, GPT_CRITIC_API_KEY, the Azure pair, and an optional TAVILY_API_KEY for search. There is no fixed vendor requirement: the README says AutoResearch does not require a fixed combination of Gemini, GPT or Claude.

Then test the routes with real requests:

set -a
. ./.env
set +a
.venv/bin/python scripts/preflight.py --live

Exit code 0 means normal roles have usable models and that multi-model stages such as Idea Forge and the critic meet their independence requirements. From there you pick a path: .venv/bin/python idea_generation.py when you have no specific idea, or the Idea Execution section of the README when you already have one.

Where the design puts work on you, and where it can fail

The provider configuration is the sharpest edge. Role assignment lives in config/providers.local.json, and the .env.example also exposes per-role overrides such as AR_MODEL_IDEATOR, AR_MODEL_PLANNER, AR_MODEL_CRITIC and AR_MODEL_CRITIC_SECONDARY, but the file itself says to prefer editing roles in the JSON for normal use. If you point every role at one model, preflight is supposed to fail the independence check, which is the correct behaviour and also a hard blocker if you only have one endpoint. The second limitation is environmental: this is a long-running stateful process with a data directory, a log directory and an output directory, and the README's own quickstart assumes a Linux or SSH machine. There is no documentation of a Windows path, no container image in the top-level layout, and no release artifacts, so you are tracking main. Third, the README is explicit that the system cannot guarantee every conclusion is correct. It preserves evidence and state for review. If you need a guaranteed-correct answer rather than an auditable trail, this is the wrong tool.

AutoResearch compared with a general coding agent

A general-purpose coding agent such as Claude Code, which the README's own ecosystem references and which appears throughout search interest in this project, will happily write experiment code in a repository and iterate on it. The difference is what surrounds the code. A coding agent has no built-in notion of a research signal, no pilot stage that gates a full run, no persisted experiment queue, and no requirement that a critic be a different model identity from the author. AutoResearch adds those as pipeline stages rather than as prompting conventions. The trade-off runs the other way too: a coding agent starts working in seconds with one credential, while AutoResearch asks you to configure multiple endpoints and pass a live preflight before it will do anything useful. If your task is writing a script, the coding agent wins. If your task is producing a defensible experimental record, the extra setup is the product.

Maintenance, licence and the cost of tracking main

The repository is not archived and the last push was on 2026-09-16, so the code is current as of that date. No releases were retrieved, which means there is no tagged version to pin and upgrades mean pulling main and re-running bash scripts/bringup.sh against your existing .venv. The bringup script runs the baseline tests, so a broken upgrade should surface there rather than mid-experiment. Budget for API spend rather than infrastructure: the pipeline makes real calls across multiple models, AR_LLM_TIMEOUT defaults to 900 seconds per call in .env.example, and a full run with pilots, critics and blind reviews multiplies that across stages. The licence is Apache-2.0, which permits commercial and closed use and includes a patent grant; the repository ships a LICENSE file and a CITATION.cff, so check the citation requirements if you publish work produced with it. Nothing here is legal advice.

Editorial conclusion

Adopt AutoResearch if you already have API credentials for at least three distinct models, a Linux or SSH machine, and a research question you are willing to see fail. Skip it if you want a one-command paper generator, if you only have a single model endpoint, or if you cannot host a long-running stateful process. Before committing, run bash scripts/bringup.sh and confirm the baseline tests pass, then run .venv/bin/python scripts/preflight.py --live and check that exit code 0 is returned, because that exit code is the project's own test of whether your multi-model independence requirement is actually satisfied.

Frequently asked questions

Official sources

1. EvoMap/AutoResearch on GitHub
2. Issues
3. License: Apache-2.0
4. README