Auto-Empirical Research Skills: a 1,096-skill catalogue for social science agents, with its own grading harness
🔬 A curated collection of 23,000+ agent skills for empirical research across 8 social science disciplines. | 精选 23,000+ AI Agent 技能库，覆盖8大社会科学学科的实证研究。CoPaper.AI 20分钟完成一篇可复现的规范实证论文，并支持用户上传 Skills。-- Maintained by CoPaper.AI from Stanford REAP.
At a glance
- What is it?
- AERS is a curated agent-skill library for empirical social science, plus a nine-stage paper pipeline and a scoring tool that lets outside agents run the same benchmark. The catalogue is the real product; the pipeline is the vendor's own workflow.
- Who is it for?
- Adopt AERS if you already work in Claude Code or Codex and want vetted, method-named skill folders for econometrics, panel data or robustness checks that you can copy into .claude/skills/ and read before use. Do not adopt it if you need a guarantee that a generated paper is correct, or if you cannot inspect the individual SKILL.md files; the README's own trust table reports only 9 of 42 eval scenarios have proven pass/fail discrimination.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly Stata, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem AERS addresses: method selection, not model capability
A general-purpose coding agent can write a regression. What it usually cannot do is pick Callaway-Sant'Anna over two-way fixed effects when treatment timing is staggered, or know that a robustness table needs HonestDiD bounds attached. That knowledge lives in papers, not in the model's default behaviour. AERS packages it as named skill folders that an agent loads on demand. The repository describes itself as a curated collection for empirical research across eight social science disciplines, with the catalogue at catalog/skills.json as the source of truth for collection and skill counts. The intended user is a researcher or graduate student who already knows the method they want and wants the agent to execute it consistently, not someone looking for the agent to invent the identification strategy. The topic list (economics, political science, sociology, psychology, education, public administration, international relations, communication) is broad, but the primary language of the repository is Stata, which tells you where the maintainers' own defaults sit.
What actually ships: skill folders, a catalogue file, and a pipeline table
A skill is a folder containing a SKILL.md file. The README warns that in some collections the SKILL.md sits one level down, and that you must copy that level instead. Agents select a skill by reading its description field, so the install is not a package resolution problem; it is a file placement problem. Three scopes are documented: a .claude/skills/ directory inside the current working directory for a single session, the same path at the project root so it can be committed to git, and ~/.claude/skills/ for global availability, with ~/.codex/skills/ given as the Codex equivalent. The README also points to docs/search.html for filtering the catalogue by method, stage, language and license, and states that make quickstart prints a five-minute tour in the terminal. The nine-stage pipeline (topic refinement through submission) is presented as a table mapping each stage to specific skill collections. That table is the maintainers' recommended path, not a constraint: the README says any intermediate step can be taken over by hand and the pipeline resumed.
Installation paths: plugin marketplace versus copying folders
Two installation routes are documented. The plugin route requires Claude Code v2.1 or later and installs named bundles:
claude plugin marketplace add brycewang-stanford/Auto-Empirical-Research-Skills claude plugin install aer-skills@auto-empirical-research-skills claude plugin install empirical-analysis-python@auto-empirical-research-skills claude plugin install empirical-analysis-stata@auto-empirical-research-skills claude plugin install empirical-analysis-r@auto-empirical-research-skills
The bundle names map to the pipeline rather than to individual methods: aer-skills is described as the full submission workflow with 9 skills, while the three empirical-analysis bundles are the Python, Stata and R plus Quarto econometrics pipelines. The manual route clones with submodules and copies a folder:
git clone --recurse-submodules https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills.git cd Auto-Empirical-Research-Skills cp -R skills/00.1-Full-empirical-analysis-skill_Python .claude/skills/
The --recurse-submodules flag matters because some collections are submodules; a plain clone leaves empty directories. The README also documents a third path where you paste the repository URL into Claude Code or Codex and state the scope in natural language, which the agent then resolves itself. That is convenient and also the least inspectable option, since you are trusting the agent to place files correctly.
The trust surface is unusually explicit, and it is small
Most skill collections assert quality. This one publishes counts. The README's trust table reports 19 numeric benchmark tasks whose gold values are recomputed from real data on each run, 42 behavioural eval scenarios containing 217 rubric items, and 9 scenarios that have been shown to discriminate correct from incorrect output using paired pass/fail fixtures, with all six critical scenarios included in that nine. Read those numbers as a boundary rather than a headline: 9 of 42 scenarios carry demonstrated discrimination. The remaining 33 are scored but not yet proven to separate right from wrong. A separate aers-score tool is offered for external grading. The documented flow is pip install -e . followed by running aers-score against your own agent, with results published in docs/EXTERNAL_SCOREBOARD.md under rules in docs/SCOREBOARD_RULES.md. The README states that leaderboard numbers are recomputed by the maintainers with the same scorer rather than self-reported by submitters. That is a meaningful design choice, and it also means the scoreboard only contains agents someone chose to submit.
Where AERS is the wrong tool
The licence is the first constraint. The repository metadata reports NOASSERTION while the README badge and badge link point to CC BY-SA 4.0. Those two signals disagree, and CC BY-SA carries a share-alike obligation that matters if you plan to redistribute modified skills inside a proprietary product. Confirm the actual terms before that use; this is not a legal opinion, just a flag that the metadata and the badge do not match. The second constraint is scope. Stata is the primary language, so an R-first or Julia-first workflow will be using a minority path. Third, the eval harness grades behaviour against rubrics, not statistical validity. An agent can pass a scenario about producing a robustness table while the underlying specification is wrong for the data. Nothing in the supplied material claims the harness detects that. Fourth, the nine-stage pipeline is the vendor's own workflow: the repository is maintained by CoPaper.AI, the homepage is copaper.ai, and the pipeline is described as ending in a submittable paper. If you want a skill library and not a paper generator, use the individual skill folders and ignore the pipeline table.
How this differs from a general-purpose agent skill collection
The obvious comparison is a broad community skill marketplace, where skills accumulate without a stated validation gate. AERS differs in two concrete ways. First, contributions are gated: the README instructs contributors to run make check locally, described as catalogue validation plus link checking plus unit tests plus the eval harness plus the benchmark, and a validate-catalog GitHub Action enforces catalogue consistency. Second, statistics are machine-checked. The README states that collection and skill counts come from catalog/skills.json and are guarded by a readme-stats checker inside make validate. A general marketplace typically has neither a catalogue file nor a numeric gate. The trade-off is the opposite of what you might expect: AERS is narrower and more opinionated, and its opinion is a Stata-centric, econometrics-first view of social science. If your field's conventions do not match that view, the validation work the maintainers did does not transfer to your problem.
Maintenance, versioning and what to verify before you commit
There is one tagged release, v2026.07, described as the first tagged release, dated 2026-07-02. The repository is not archived and the last push recorded is 2026-09-07, so activity continued after the tag. Documentation is maintained in two layers: the README holds the pipeline overview and a 76-row collection table, while docs/CONTENT_ZH.md holds per-collection detail, grouping and exact numbers. The README states a maintenance rule that edits to the collection table must be synced in both files, and that detail changes go only to CONTENT_ZH.md. That rule is a real cost: two files to keep aligned, guarded by the readme-stats checker. Note also that the default entry point is Chinese, with README-en.md as the English version and README-zh-CN.md deprecated to a redirect placeholder. If your team works in English, you will be reading the translated file, and translation drift between the two is a plausible failure mode the repository does not appear to guard against. Before committing to AERS, run make check on a fresh clone, then pip install -e . and run aers-score against your own agent to see where it lands on the published benchmark.
Editorial conclusion
Adopt AERS if you already work in Claude Code or Codex and want vetted, method-named skill folders for econometrics, panel data or robustness checks that you can copy into .claude/skills/ and read before use. Do not adopt it if you need a guarantee that a generated paper is correct, or if you cannot inspect the individual SKILL.md files; the README's own trust table reports only 9 of 42 eval scenarios have proven pass/fail discrimination. Before relying on anything, run make check locally, then pip install -e . and run aers-score against your own agent to see how it performs on the same benchmark the maintainers publish.
Community notes