AI Can Learn Scientific Taste: RLCF, Scientific Judge and Scientific Thinker
We propose Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale community signals as supervision, and formulate scientific taste learning as a preference modeling and alignment problem.
At a glance
- What is it?
- A research release that turns citation behaviour into pairwise preferences and trains a generative reward model on them. The artifacts are models, datasets and a paper, not a runnable pipeline you can install and point at your own corpus.
- Who is it for?
- Adopt this if you are building an evaluator or an idea generator for arXiv-style research proposals and you want a released reward model plus a 720,341-pair preference dataset to start from. Do not adopt it if you need a maintained library, a CLI, or a way to score papers outside Computer Science, Mathematics, Physics and the biology material covered by the bioRxiv transfer test, because the repository publishes models and data rather than an application.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 55 days ago.
- What is it written in?
- GitHub does not report a main language for this repository.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem: peer review and citation counts are both bad supervisors
Judging whether a research idea is worth pursuing is normally done by people who have spent years in a field. The project's abstract calls this scientific taste: the ability to judge and propose research ideas with potential for long-term scientific impact. The gap it targets is that this judgement is expensive to obtain and hard to turn into a training signal. Peer review gives you a label, but a coarse one, produced by a handful of reviewers on a schedule. Citation counts give you a number, but only after years have passed, and they are confounded by field, venue and author visibility.
The audience is narrow but real. It is people training models to rank or generate research ideas, and people who want a reward model for open-ended scientific text where a correct answer does not exist. If your task has a verifiable answer, a string match or a unit test, this is the wrong tool and you should use that instead. RLCF exists for the case where the only ground truth available is what a community eventually rewarded.
How RLCF turns citations into pairwise preferences
The mechanism has three stages, and the first one is where most of the design risk sits. Citations are converted into pairwise preference signals by matching papers within the same field and publication period. Two papers enter, one is treated as preferred because it accumulated more citations, and the pair is passed downstream as a preference example.
Matching on field and time is the part that makes the signal usable. Comparing a 2019 systems paper against a 2024 theory paper would mostly measure age and subfield popularity. The README states the pairs are field- and time-matched, so the comparison is meant to isolate impact within a comparable cohort.
Stage two trains a generative reward model, Scientific Judge, with GRPO. It reasons over a pair of paper abstracts and predicts which one is more likely to have higher impact. It is generative rather than a scalar head, so the model produces reasoning before the preference decision. Stage three uses that judge as the reward model and optimizes Scientific Thinker, a policy that takes a paper title and abstract and proposes follow-up research ideas, trained with comparison-based GRPO.
The data flow is therefore: arXiv papers to matched pairs to a judge that scores pairs to a policy that generates ideas the judge prefers. The judge is both an evaluator and the training signal, which is the standard weakness of reward-model-based alignment and is worth keeping in mind when reading the win rates.
SciJudgeBench: what the 720,341 pairs actually contain
SciJudgeBench is described as 720,341 preference pairs and 1,440,682 pair-level paper records. The README is explicit that papers may recur across pairs, so the second number is not a unique-paper count. That distinction matters if you plan to fine-tune on it: a small set of frequently cited papers could appear in many pairs, and naive splitting will leak the same abstract across train and test.
The corpus is built from arXiv papers across Computer Science, Mathematics, Physics and an Other category. Evaluation goes beyond the in-domain split: temporal OOD on 904 pairs from papers published in 2025, metric OOD using ICLR peer review and Altmetric attention, field transfer, and controlled comparisons, with bioRxiv as an additional biology evaluation. The news section notes the 2025 Future-Year benchmark and evaluation results were refreshed to 904 pairs in July 2026.
Using ICLR peer review and Altmetric as out-of-distribution metrics is the more interesting choice here. It tests whether the judge learned something about impact rather than about the citation counter specifically. The project reports that gains persist under author and institution and topic controls, which is the result to check first if you are deciding whether to trust the signal, because citation-based preference learning is exactly the setting where prestige confounds are easiest to absorb by accident.
Reported results, and how to read the baseline comparison
The headline numbers in the README are: Scientific Judge-Qwen3-30B at 82.7% in-domain accuracy, against GPT-5.4 Thinking at 81.6%. On the refreshed 904-pair 2025 test set, Qwen3-4B improves from 64.7% to 80.9% and Qwen3-30B-A3B improves from 71.7% to 83.1%. Scientific Thinker is reported at a 54.2% average win rate against three strong LLM baselines in in-domain and out-of-domain settings, against 30.3% and 27.8% for its base policy.
The judge result is a 1.1 point margin over the listed baseline. On a benchmark of this kind that is close enough that the evaluation protocol matters more than the ranking, and the README does not describe how baseline prompts were tuned or how many samples were drawn. The temporal OOD gains are the more persuasive figure, because a 16.2 point improvement over the same base model on future-year papers is harder to explain by prompt sensitivity.
The Thinker number needs a different reading. A 54.2% win rate against three baselines is a narrow margin over an even split, and the reward model used to train the policy is the same model family being compared. Win rates judged by a model that shares training lineage with the policy are not independent evidence. Treat the judge accuracy as the claim to scrutinise and the Thinker win rate as suggestive.
There is no install: what the repository ships
This is the main practical constraint. The material shows no package, no CLI, no training script and no configuration keys. The repository is a project page plus a README that links to an arXiv paper, a Hugging Face collection named OpenMOSS-Team/ai-can-learn-scientific-taste, an online demo at paperank.open-moss.com, and a LICENSE file. There are no releases retrieved, and the primary language is listed as unknown.
So the honest onboarding path is: read the paper, pull the SciJudge-2506 series models and the SciJudgeBench dataset from the Hugging Face collection, and write your own training and inference code around them. The README gives a BibTeX entry for the paper, which is the citation form to use if you build on it. It does not give an example command, a config file, or an inference snippet, and I am not going to invent one.
If you need a drop-in evaluator, this will cost you engineering time before it returns anything. If you are already set up to fine-tune Qwen-family models with GRPO and you have your own preference data, the released judge is a useful starting checkpoint and the dataset is a useful reference for how to construct pairs.
Where the approach breaks: domain coverage and citation lag
Two limitations follow directly from the design. The first is coverage. SciJudgeBench is built from arXiv across Computer Science, Mathematics, Physics and Other, with bioRxiv added as a biology evaluation set. Fields where the preprint culture is weak, or where impact shows up in patents, clinical uptake or policy rather than citations, are not represented in the described corpus. Running the judge on a humanities or social science abstract is outside the tested distribution, and the README gives no evidence about behaviour there.
The second is citation lag. The preference signal comes from citations, which means the label for a recent paper is incomplete. The project handles this by matching within publication period and by testing temporal OOD on 2025 papers, but a paper published last month has not had time to accumulate the citations that define its label. Any pipeline that scores very recent work inherits that noise, and no amount of field and time matching removes it.
There is also a subtler failure mode. The judge is trained to predict which of two abstracts is more likely to be highly cited. That is a proxy for impact, not impact itself. A model optimised against this proxy will reward abstracts that look like the ones the community historically cited, which can select for conventional framing over genuinely novel work. The README does not report a test for this, and it is the question I would want answered before using the judge as a filter on submissions or proposals.
Alternatives: pairwise preference data versus scalar reward models
The obvious comparison is a standard scalar reward model trained on human preference labels, the kind used in RLHF pipelines. The difference is in the supervision source and the output form. A scalar reward model compresses a judgement into one number, and its labels come from annotators paid to compare outputs. RLCF's judge is generative, produces reasoning over a pair of abstracts, and its labels come from citation behaviour aggregated across a community rather than from individual raters.
That trade is real in both directions. Community signals are cheap and available at a scale no annotation budget reaches, which is what makes 720,341 pairs possible. They are also noisy, delayed, and correlated with prestige. Human labels are expensive and slow but can be collected on questions citations cannot answer, such as whether a research direction is ethically sound or whether a result is reproducible.
A second comparison is to bibliometric ranking, the thing this project is effectively learning to imitate. A citation-count baseline needs no training and no GPU, and it is transparent. The reason to prefer a learned judge is transfer: the README reports that judgement transfers to future-year papers, unseen fields and other community metrics including ICLR peer review and Altmetric. If that transfer holds on your data, the learned judge gives you something a citation counter cannot, which is a score for a paper that has not been cited yet.
Licence, maintenance and what to verify before committing
The repository is Apache-2.0, and the README points to the LICENSE file for details. Apache-2.0 covers the repository contents and includes a patent grant, which is friendlier than a bare MIT for a research artifact. It does not automatically tell you the terms attached to the Hugging Face model weights or the dataset, since those are separate distributions with their own cards. Check the collection page for each artifact rather than assuming the repository licence carries over. This is a description of what the files say, not legal advice.
The maintenance picture is a research release, not a product. Last push is July 2026, there are no tagged releases, and the July 2026 news items are a benchmark refresh and a SciJudge-2506 model update. That is a reasonable cadence for an academic project and a poor one if you need a dependency with a changelog. The realistic upgrade cost is that a new SciJudge version may shift the score distribution, so any threshold you calibrate on the current judge needs re-checking against the new one.
Before adopting, verify three things: whether the Hugging Face model card states the base model and training configuration in enough detail to reproduce the reported accuracy, whether SciJudgeBench ships with the field and time matching metadata so you can build non-leaking splits, and whether the controlled-comparison results in the arXiv paper hold when you re-run the judge on your own field. The last one is the only test that answers whether this transfers to your problem.
Editorial conclusion
Adopt this if you are building an evaluator or an idea generator for arXiv-style research proposals and you want a released reward model plus a 720,341-pair preference dataset to start from. Do not adopt it if you need a maintained library, a CLI, or a way to score papers outside Computer Science, Mathematics, Physics and the biology material covered by the bioRxiv transfer test, because the repository publishes models and data rather than an application. Before relying on it, check the Hugging Face collection for the actual model and dataset cards, confirm which licence file applies to the weights as opposed to the code, and read the controlled-comparison section of the arXiv paper to see how much of the reported gain survives author and institution controls.
Community notes