PaperOrchestra turns the arXiv paper's five-agent pipeline into a skill pack your coding agent executes
An automated AI research-paper writer based off Google's PaperOrchestra paper's implementation through a skills - benchmark + autoraters using any coding agent (Claude Code, Cursor, Antigravity, Cline, Aider). No API keys, no LLM SDKs.
At a glance
- What is it?
- Ar9av/PaperOrchestra is a Python skill pack that ports the prompts, schemas and halt rules from the PaperOrchestra paper into instruction files any coding agent can follow. It ships no LLM SDK and no API keys, which is the whole point and also the main constraint.
- Who is it for?
- Adopt PaperOrchestra if you already work inside a coding agent, have a raw experimental log and a LaTeX template, and want the paper's five-step pipeline without wiring up API credentials. Skip it if you need a standalone CLI that runs unattended, since every LLM call and every Semantic Scholar lookup is delegated to the host agent by instruction.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 11 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- Ar9av/PaperOrchestra717 ★ · +63
| Repository | 2026-09-13 | 2026-11-15 |
|---|---|---|
| Ar9av/PaperOrchestra | 654 | 717 |
63 Stars in the last 63 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The gap between a paper's prompts and a pipeline you can run
The PaperOrchestra paper defines five agents in sequence: Outline, Plotting, Literature Review, Section Writing, Content Refinement. The README states the paper ships the exact prompts for every agent in Appendix F, and that the pipeline outperforms single-agent and tree-search baselines on the PaperWritingBench benchmark. Prompts in an appendix are not a program. You cannot point a shell at Appendix F. This repository exists to close that distance, and it makes a specific choice about how: rather than reimplementing the agents as Python classes that call a model, it converts the prompts, schemas, halt rules and verification pipelines into instruction documents a coding agent reads and follows. The audience is therefore narrow and identifiable. You need a research artifact (an idea and an experimental log), a LaTeX template, and a coding agent that can read files, run local scripts, search the web and compile LaTeX. If you have a headless batch job in mind, this is not built for you.
SKILL.md plus deterministic scripts: the actual division of labour
Each skill in the pack has the same three-part shape. SKILL.md is described as a dense instruction document the host agent reads and follows. A references/ directory holds verbatim paper prompts from Appendix F, JSON schemas, rubrics, halt rules and example outputs. A scripts/ directory holds helpers the README calls purely deterministic: JSON schema validation, Levenshtein fuzzy matching, BibTeX formatting, deduplication, LaTeX sanity checks and coverage gates. The scripts have no network access, no LLM and no API keys. Everything requiring judgement (LLM reasoning, web search, Semantic Scholar lookups, LaTeX compilation) is delegated to the host agent by instruction. That split is the design. It means the repository can stay small and auditable, and it means the quality ceiling is set by your agent, not by this code. The reference file skills/paper-orchestra/references/host-integration.md is where per-host invocation is documented for Claude Code, Cursor, Antigravity, Cline and Aider.
Seven skills, their call budgets, and the parallel step
The README lists seven skills plus an optional eighth. paper-orchestra is the orchestrator that coordinates the rest. outline-agent handles Step 1 with a single LLM call, turning an idea, a log, a template and guidelines into structured outline JSON containing a plotting plan, a literature review plan and a section plan. plotting-agent covers Step 2 at roughly 20 to 30 calls, executing the plotting plan, rendering plots and conceptual diagrams, optionally running a VLM-critique refinement loop, and captioning everything. literature-review-agent covers Step 3 at roughly 20 to 30 calls: web-search candidates, verify against Semantic Scholar with a Levenshtein threshold above 70 plus a cutoff and deduplication, then draft the Introduction and Related Work with at least 90 percent citation integration. section-writing-agent is a single multimodal call that drafts the remaining sections, builds tables from the experimental log and splices in figures. content-refinement-agent runs simulated peer review across roughly 5 to 7 calls, accepting or reverting per strict halt rules, with safety constraints intended to stop the agent gaming the evaluator. paper-writing-bench reverse-engineers raw materials (Sparse or Dense idea, experimental log) from an existing paper to build benchmark cases, and paper-autoraters runs the paper's own autoraters: Citation F1 at P0 and P1, a six-axis LitReview quality score, and side-by-side paper and litreview quality comparisons. Steps 2 and 3 run in parallel, per the pipeline reference. The call counts are the README's own estimates, and they are the number that determines your cost.
Getting it running: symlinks, search roots and two input files
The README gives one concrete install command, and it is a symlink into your host's skill directory:
ln -sf ~/paper-orchestra/skills/agent-research-aggregator ~/.claude/skills/agent-research-aggregator
For Cursor, Antigravity, Cline and Aider the README points you at the host-integration reference rather than repeating the command, so treat that file as required reading before you assume your host is supported the same way. The aggregator skill has no dependencies beyond the base requirements.txt. Its discovery phase is driven by a --search-roots flag on discover_logs.py, which walks those roots to catalogue relevant log files across agent caches and prints a summary for review before anything is read. The extraction phase applies references/extraction-prompt.md to batches of roughly 50 KB and produces raw_experiments.json, strips PII and flags unverified numbers with an [UNVERIFIED] marker. Synthesis merges records into synthesis.json and pauses to ask the user if it detects multiple disconnected projects. Formatting runs format_po_inputs.py to emit workspace/inputs/idea.md in the Sparse Idea format from section 3.1 of the paper and workspace/inputs/experimental_log.md following Appendix D.3. The aggregator is optional: if those two input files already exist, it skips itself.
The failure mode is your agent, not the scripts
Because the pipeline has no LLM of its own, every failure that matters is a host-agent failure. If your agent's web search is weak, the literature-review step has weak candidates to verify, and the Levenshtein threshold above 70 will reject them rather than rescue them. If your agent cannot compile LaTeX, the scripts' sanity checks will flag a problem the pipeline cannot fix. The README states that the aggregator flags unverified numbers as [UNVERIFIED], which is honest but also a warning: the pipeline will carry those numbers into idea.md and experimental_log.md, and from there into tables built by section-writing-agent. Nothing in the deterministic scripts can tell whether a number is true. The other limitation is structural. Running this inside a coding agent means the run is interactive and session-bound. A long plotting or literature pass consumes the host agent's context, and the README's own call estimates (roughly 20 to 30 calls for each of those two steps) suggest that budget is real. If your materials are confidential, note that the aggregator's extraction phase sends batches to your host agent's model, and the literature step performs web searches; the scripts themselves do not, but the pipeline around them does.
How this differs from a Python framework that calls the model itself
The obvious alternative is a framework that implements the multi-agent pipeline directly: Python code that holds the API keys, calls the model, and runs to completion without a human present. The difference is not cosmetic. Such a framework can define prompts, model settings and execution flow in code, but reproducibility still depends on the model, inputs and external services. In this design the prompts live in references/ as instruction documents, the model is whatever your agent is configured to use, and the reasoning path varies with the agent. You gain portability across Claude Code, Cursor, Antigravity, Cline, Aider and OpenCode, and you avoid managing credentials entirely. Behavior and unattended operation depend on the host agent and its configuration. For a paper draft that a human will read and revise, the trade is reasonable. For a service that must produce the same output twice, it is not. A second alternative is simply writing the paper yourself with an agent assisting on individual sections. That keeps the judgement where it belongs and costs nothing to set up, but you lose the halt rules and the autoraters, which are the parts of the paper this repository actually ports.
Maintenance, versioning and the licence question
The repository has two releases: v0.1.0 on 2026-04-09 and v0.2.0 on 2026-04-25, the latter described as adding multi-agent skills, PaperBanana and new search backends. The last push recorded is 2026-08-09, and the repository is not archived. Because the skills are instruction documents, upgrades are textual: a change to a SKILL.md or a reference prompt changes agent behaviour without any dependency bump, which makes version pinning more important than usual. If you vendor the pack, pin to a release tag rather than tracking main. The current README identifies the repository license as MIT. Review the LICENSE file and any third-party material before redistributing the skill pack. The paper itself is on arXiv and is cited in the README.
Editorial conclusion
Adopt PaperOrchestra if you already work inside a coding agent, have a raw experimental log and a LaTeX template, and want the paper's five-step pipeline without wiring up API credentials. Skip it if you need a standalone CLI that runs unattended, since every LLM call and every Semantic Scholar lookup is delegated to the host agent by instruction. Before trusting it, read skills/paper-orchestra/references/host-integration.md for your specific host and confirm the deterministic scripts under scripts/ actually pass on your inputs. The current README identifies the license as MIT; check the repository's LICENSE file and any third-party material before redistribution.
Frequently asked questions
Does PaperOrchestra need an API key or an LLM SDK?
No. It ships no API keys, no SDK dependencies and no embedded LLM calls. The only Python requirements are `jsonschema`, `python-Levenshtein`, `reportlab`, `matplotlib` and `pypdf`, all used by deterministic helpers, while your coding agent makes every model call and web search.
How many LLM calls does the PaperOrchestra pipeline make?
Outline takes one call and section writing takes one multimodal call, while plotting and literature review take roughly twenty to thirty each and content refinement takes roughly five to seven. Steps two and three run in parallel, so the two expensive steps overlap.
How does PaperOrchestra verify the citations it finds?
The literature review skill checks candidates against Semantic Scholar with a Levenshtein fuzzy match above 70, a cutoff and deduplication, then targets at least 90% citation integration when drafting the Introduction and Related Work.
What does the PaperOrchestra agent-research-aggregator do when its input files exist?
It skips itself. If `workspace/inputs/idea.md` and `workspace/inputs/experimental_log.md` are already present the pipeline proceeds directly, and the skill only runs when the inputs are missing or you point an agent at a directory.
What is the newest PaperOrchestra release and what did it add?
v0.3.0, tagged 2026-09-21, titled deterministic quality gates and the section-rhetoric layer. It followed v0.2.0 on 2026-04-25, which added multi-agent skills, PaperBanana and new search backends.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/ar9av-paperorchestra)Star history chart for your README
[](https://hysenlabs.com/projects/ar9av-paperorchestra)Was this useful?