AutoScholarLoop: a staged multi-agent research loop with checkpoints

Five role-based stages instead of one chatbot

AutoScholarLoop is described as an open-source AUTO Research framework for building auditable, multi-agent research loops. The framing that matters is that it simulates a small research group rather than a single chatbot, and the readme is explicit that the system separates research work into role-based stages.

The five stages are named after the roles they mimic. S00, the Field Archive Group, builds the field map, the paper cards, the method map, the dataset and baseline map, and the evidence bank. S01, the Professor Decision Group, runs multi-round idea generation, then critique, novelty probing, ranking and direction selection. S02, the PhD Execution Group, plans baselines, handles implementation, runs experiments, does failure analysis and produces professor review memos. S03, the Writing Group, turns the evidence into a manuscript plan, a claim-evidence table, a draft and figures, then revises in response to review. S04, the Quality Control Group audits novelty, citations, reproducibility, unsupported claims and final release readiness.

The nested loop that ties them together is drawn in the readme as a short text diagram:

S00 evidence preparation
  -> S01 professor decision loop
  -> S02 execution-review loop
  -> S03 writing-review loop
  -> S04 quality gate
  -> submission_candidate / revise / pivot / kill

That last line is the piece worth dwelling on. The loop does not have one success path. A run can end as a submission candidate, it can be sent back for revision, it can pivot to a different direction, or it can be killed. The set of outcomes mirrors what a real research group does when an idea fails, and it means the system is designed to be stopped rather than to always produce a paper.

The project is developed for research automation scenarios at AI Group, CAS CNIC, the Computer Network Information Center of the Chinese Academy of Sciences, and it is described as part of BoundX, Yisuan Fangzhou. The package author in the project metadata is CAS CNIC, Chinese Academy of Sciences.

Every stage leaves a checkpoint you can read

The auditability claim rests on a concrete mechanism: every stage writes Markdown checkpoints and structured artifacts, so a user can inspect how an idea was created, why it was selected, what evidence supports the paper, and where the quality gates passed or failed.

Each run creates a workspace directory with a fixed layout:

run/
  source_papers/
  inputs/
  artifacts/
  logs/
  code/
  00_field_context/
  01_decision/
  02_execution/
  03_writing/
  04_quality/
  paper/
  release/

The output files are named just as precisely. The execution stage leaves code/experiments/run_experiment.py, code/methods/proposed_method.py and code/experiments/result.json. The field context stage leaves field_map.md and paper_cards.md. The decision stage leaves IDEA_REPORT.md and chosen_direction.md. The execution stage leaves RESULTS_ANALYSIS.md, CLAIMS_FROM_RESULTS.md and EXPERIMENT_AUDIT.md. The writing stage leaves PAPER_PLAN.md and claim_evidence_table.md. The quality stage leaves CITATION_AUDIT.md, final_gate.md and compile_report.md, and the paper directory ends with final_draft.md and main.tex.

Two of these names carry most of the weight. The claim-evidence table and CLAIMS_FROM_RESULTS.md are the bridge between what was measured and what is asserted in the manuscript, and the EXPERIMENT_AUDIT.md and CITATION_AUDIT.md files are the checks on that bridge. A system that writes papers from model output alone is hard to trust; a system that writes papers and also writes down which result supports which sentence is a different proposition.

The naming convention also means a run is inspectable without reading any code. A reviewer can open final_gate.md and see whether the quality stage passed, and open claim_evidence_table.md to see whether the manuscript's claims are traceable to the run's results.

Running it from the command line

Installation is a single editable install with extras:

cd AutoScholarLoop
pip install -e ".[api,web,dev]"

The extras map onto the features. api pulls openai 1.0.0 or newer for the OpenAI-compatible provider adapter, web pulls fastapi 0.110.0 or newer, uvicorn with the standard extra, python-multipart and pypdf 4.0.0 or newer, and dev pulls pytest 8.0.0 or newer. The base package itself declares no dependencies at all, and requires Python 3.10 or newer.

The quickest run takes a research direction as a seed and names a workspace:

autoscholarloop run `
  --seed "I want to study retrieval-augmented agents for scientific writing." `
  --loop-mode fast `
  --paper-format ieee `
  --workspace runs/demo

The fuller set of options is the one to read before a serious run:

autoscholarloop run `
  --seed "your research idea" `
  --reference "paper title, URL, local path, or note" `
  --num-ideas 5 `
  --loop-mode standard `
  --paper-format acm `
  --literature semanticscholar `
  --execution-backend dry-run `
  --review-ensemble 5 `
  --compile-pdf `
  --workspace runs/demo

The flags map onto the stages. --num-ideas sets how many candidates the decision group generates, --loop-mode chooses between fast and standard, --review-ensemble sets how many reviewers take part, --literature selects the adapter, and --execution-backend selects dry-run or local shell execution. --reference accepts a paper title, a URL, a local path or a note, which is how prior work gets into the field archive.

The legacy command alias new-ai-scientist is kept for compatibility, and the package metadata declares both entry points pointing at the same cli:main function.

One thing to know before you start: by default the local provider mode is a deterministic demo, and the readme states plainly that it is not a real model-backed research run. Real runs need an OpenAI-compatible provider and API key configured.

Providers, literature adapters and the local shell backend

The provider layer has two halves. There is a deterministic local provider for offline demos and tests, and there is an OpenAI-compatible provider adapter for real model APIs.

The OpenAI-compatible route points at any service that speaks the OpenAI protocol, and the readme gives a DeepSeek-compatible example, including the environment variables and the endpoint:

$env:OPENAI_API_KEY='your_api_key'
$env:AUTOSCHOLARLOOP_HTTP_TRUST_ENV='0'
autoscholarloop run `
  --seed "your research idea" `
  --provider openai-compatible `
  --model deepseek-chat `
  --base-url https://api.deepseek.com/v1 `
  --workspace runs/deepseek_demo

The AUTOSCHOLARLOOP_HTTP_TRUST_ENV variable is the kind of detail that saves an afternoon. If a system proxy is injected into the environment and breaks httpx connections, setting that variable to 0 makes httpx ignore the inherited proxy settings. The same bypass is offered in the Web console as an option.

On the literature side there are three adapters: local, Semantic Scholar and OpenAlex. The local one reads from files you supply, and the CLI's --reference flag is what feeds it, accepting a title, a URL, a local path or a note.

The execution backends are dry-run and local shell. The readme is explicit that the Web console defaults to local shell execution, where model-generated code is written under code/ and then run on the current machine. That is a significant design choice rather than a footnote, and it is the point at which the framework stops being a document generator and starts running things. The code the model writes lands in code/experiments/run_experiment.py and code/methods/proposed_method.py, and the results come back as code/experiments/result.json.

The security implication follows directly: with the local shell backend, the framework executes model-authored code on your machine with your permissions. The dry-run backend exists for the cases where you want the loop and the artifacts without that.

The Web console and the paper export path

The Web console is a Vue application that talks to a Python API, and the readme is clear that the CLI and the Web API use the same underlying research pipeline. So the console is a different face on the same engine, not a separate implementation.

The API is started with the autoscholarloop web command, which takes a host and a port:

autoscholarloop web --host 127.0.0.1 --port 8000

The frontend is started separately from the web directory with npm run dev, after npm install. The readme notes that the frontend currently uses Vite 2 for compatibility with older Node environments, and that Node 18 or newer is recommended for future frontend upgrades, which tells you the build tooling is deliberately conservative.

What the console actually offers is a list of capabilities: first-run large model API configuration, provider presets for Local demo, DeepSeek, OpenAI and custom OpenAI-compatible APIs, an optional system proxy bypass, research direction and target venue input, PDF, Markdown, text and BibTeX upload, loop mode and backend selection, manuscript format selection, live S00 to S04 progress visualisation, and a checkpoint preview for field maps, ideas, execution reports, paper plans, claim evidence, the final gate and the final draft. That checkpoint preview is the console's real value, since it is the same auditable trail described earlier, surfaced in the browser.

On the output side there are four manuscript targets: acm for an ACM-style conference or journal article draft, ieee for an IEEE one, springer_lncs for a Springer LNCS proceedings-style draft, and chinese_thesis for a generic Chinese thesis-style manuscript. Export is to Markdown and to LaTeX, and PDF compilation is optional and only attempted when --compile-pdf is enabled and a LaTeX toolchain plus the required venue class files are installed. The compiler prefers latexmk and falls back to xelatex or pdflatex with multi-pass compilation. The system always writes paper/main.tex regardless.

The readme also states plainly that official venue class files and bibliography rules still need to be checked before a real submission, and that the generated manuscript is a research draft and audit package rather than a guaranteed submittable paper.

Where AutoScholarLoop is the wrong tool

The first limitation is that the default run is a demo. The readme states that local provider mode is a deterministic demo and explicitly not a real model-backed research run. That means the pipeline, the checkpoints and the artifact layout can be evaluated without any API cost, but nothing about the quality of the research output can be evaluated that way. A team that wants to know whether the generated ideas are any good has to spend API budget to find out.

The second is the local shell execution backend, which runs model-generated code on your machine with your permissions. The readme describes it plainly: code is written under code/ and then run on the current machine. That is what makes the execution stage produce real result.json files rather than simulated ones, and it is also the sharpest edge in the project. A team handling unreviewed credentials, on a machine with access to production data, should use the dry-run backend until it has read the generated code.

The third is that the loop can end in a kill. That is a feature of the design, and it is also a limit on expectations. The pipeline will tell you that an idea was not selected, that a direction was pivoted or that the quality gate stopped the run. If what you need is a document generator that always produces a draft, this framework will occasionally refuse, and you have to accept that as a feature or work around it.

The fourth is the output's legal and academic standing. The readme says the generated manuscript is a research draft and audit package, not a guaranteed submission, and that venue class files and bibliography rules still need checking before a real submission. The citation audit is a stage in the pipeline, not a guarantee that the reference list is correct or complete. Nothing in the repository handles authorship policy, disclosure of model use, or the rules a specific venue applies to generated text.

The fifth is the maturity and licence. The version is 0.1.0, the repository publishes no GitHub releases, and the licence is one GitHub cannot classify, with LICENSE.md in the repository root rather than a standard licence identifier. The last push was on 2026-09-24. A framework that runs model-authored code and executes it locally deserves a licence and a release process you can point at before you deploy it on a shared machine.

Editorial conclusion

AutoScholarLoop fits a research group that wants an agent-driven literature-to-manuscript pipeline whose intermediate decisions can be read and challenged, since each stage writes its own checkpoints and the workspace preserves the evidence behind every claim. It is the wrong choice if you want a finished paper in an afternoon, because the deterministic local provider is described as a demo rather than a real model-backed run, and the wrong choice if you need execution that is not local shell code on your own machine. Before adopting it, read LICENSE.md, confirm the seed and reference options match the research direction you actually have, and note that the last push was on 2026-09-24 with version 0.1.0 and no GitHub releases published.

Frequently asked questions

Official sources

1. damonwan1/AutoScholarLoop on GitHub
2. Issues
3. README