Ai-Review turns a manuscript into a structured LLM review

What the repository actually contains

The repository description is brief, Large model-assisted paper review, and the directory listing shows that this is a prompts-and-templates project rather than an application.

The top level has Examples/ with the worked examples, Prompts/ with the prompt templates, ai-review-skills/ with the Skill definition, a Logo.png, a Show.gif recording the usage, and README.md alongside README_CH.md, which is the Chinese version of the same document. The LICENSE is MIT.

That division of labour is the project. The review logic is a set of prompt templates plus a few demonstration outputs, and the engine that runs them is the hosted site at ai-review.neurodong.top, which the badges say is deployed on Cloudflare. The repository is the part you can read, copy and modify.

The examples are worth looking at in detail because they show what the output is expected to look like. There is Examples/review_in_Resnet.pdf, a review of Deep Residual Learning for Image Recognition, and the same review in markdown as Examples/review_in_Resnet.md, alongside Examples/Relative_Rank_evaluation.md, which records the evaluation of the ranking mode on 100 pools taken from PeerRead ICLR 2017.

The last push was on 2026-08-25, the repository is not archived, and there are no GitHub releases, so the version history is in the commit log rather than in tags.

The four things the web app does

The usage tutorial is organised as four modes, each on its own page, and the separation is the clearest statement of scope in the README.

The first is the basic LLM review at the site root. You upload a PDF manuscript, choose a prompting mode such as SoT or Few-Shot, and get structured Strengths, Weaknesses and Suggestions back. There is a quick demo on the page, and you can configure your own API, with Deepseek given as the example, for more stable results.

The second is the VLM review. The system captures page snapshots of the PDF and runs the review over those images rather than over extracted text, so layout and figures are taken into account. The README describes it as best when you need the model to see figures and equation layout, and the scientific image quality evaluation is listed under the review methods as included, with layout awareness and image aesthetics implemented in the VLM path.

The third is prompt injection detection, and it is the feature that shows the author has thought about the failure mode. You upload a PDF and the system checks whether the file contains hidden instructions meant to steer or override an AI reviewer. The advice is to use it before relying on an AI review if you worry the PDF may have been tampered with.

That is a real problem in this setting. An AI reviewer that reads a document cannot tell the difference between an instruction in the paper and an instruction planted in the paper, so a reviewer that reads attacker-supplied text is a reader you have to trust with the contents of that text. A separate detection pass is the mitigation.

The fourth mode is Relative Rank, added on 20/08/2026. You upload your manuscript and either pick a target venue such as ICLR or NeurIPS, so the system can select comparison papers from that venue's accepted list, or upload comparison papers yourself. Every PDF is anonymized, then the pool is ranked by competitiveness.

What Relative Rank is and is not

The newest feature gets the most careful wording in the README, and that wording is the most useful thing to know before you use it.

The disclaimer is explicit: this is a relative ranking among the papers in the pool, not a conference decision or an accept or reject prediction. And the summary is the cleanest statement of the division of labour: Review tells you how to improve, Relative Rank tells you where you stand against comparable work.

So the two features answer different questions. A review is diagnostic and about your text. The ranking is comparative and about your position in a set. A tool that claimed to do both would be making a prediction it cannot support, and the README is careful not to claim that.

The measured behaviour is documented in the evaluation file. The evaluation covers 100 pools, and the README reports a pairwise accuracy of 83.8%. That is a meaningful number and a modest one: roughly six comparisons in five are ranked the way the pool's own ordering says they should be. A tool that puts your paper at the top of a pool is telling you something about the pool, and the accuracy figure is the honest limit on how much weight to put on the position.

Anonymization is part of the design rather than a policy statement. Every PDF is anonymized before the pool is ranked, which means the comparison is about the content and not about the authors, and the venue's accepted list is the reference set rather than the reviewers' opinions.

Using it as a Skill on an agent platform

The second half of the tutorial is for people who want the review inside their editor rather than in a browser tab, and it is a small piece of glue rather than a separate application.

On platforms that support Agent Skills, such as Cursor, you can use Ai-Review as a Skill and get reviews directly in the chat. The Skill definition is included in the repository:

ai-review-skills/SKILL.md

In Cursor you place the ai-review-skills folder under .cursor/skills/ in your project, or ~/.cursor/skills/ in your user directory, and then follow the platform's instructions for loading Skills.

The supported formats are listed: LaTeX with .tex, PDF with .pdf, and Word with .docx and .doc.

Triggering it is a natural language request. You say use @ai-review-skills to review @paper, or you say 审稿, 论文审稿, or review my paper, and you provide the manuscript path, for example paper/main.tex or article.pdf. The agent then calls the Ai-Review Skill to generate the review.

The value of this route is that the manuscript never has to leave the machine, and the review happens in the same editing session as the fix. For a LaTeX project that is a real workflow difference rather than a convenience, because the agent can then edit the file it just reviewed. The cost is that it depends on the platform's Skill support and on the agent's own model, so the review quality is your agent's quality rather than the hosted service's.

The prompt engineering behind the output

The Prompt Engineering section is where the actual method lives, and it is where a user can improve the tool.

Reverse Prompt is included, and the description is unusual: infer prompts from excellent results generated by large models, with the excellent results coming from the AI review at AAAI2026. In other words the templates were recovered by working backwards from review output that was considered good, rather than written from scratch.

Few-Shot Prompting is included, and in the web UI you select Prompt + Examples (Few-Shot) under Prompting mode. The system loads templates from the Prompts/ directory and provides the examples Examples/review_in_Resnet.md and Examples/review_in_Verified.md as examples to the model. The second example is the one with the verified output, and having both a plain and a verified version in the same directory tells you the authors treat the good output as a fixture worth keeping.

Chain-of-Thought Prompting is included, allowing the model to explicitly perform step-by-step reasoning before answering, with the templates in the Prompts/ directory.

One item is listed as planned rather than done: dynamic interactive prompts, where the large language model proposes a plan first, interacts with users, and then proceeds with the review after the plan is finalized. That is the obvious next step for a tool whose weakness is that a single pass over a manuscript cannot ask a clarifying question about a missing experiment.

The continuous update section says the repository will be updated continuously and invites issues and pull requests to improve the prompt templates and website features. That is the mechanism by which the method improves: the templates are the contribution, and anyone can propose better ones.

Where Ai-Review fits, and where it stops

The comparison worth making is with the other two options for a machine-assisted read of your manuscript: a general purpose chat assistant you paste the text into, and a human reviewer.

Against a chat assistant, the difference is the prompt templates and the output structure. A general assistant will read a PDF if you give it one, and it will produce something review-shaped, but it will not apply a consistent structure across dozens of manuscripts, and it will not have been tuned against a known-good output. Ai-Review's contribution is the reverse-prompted templates and the fixed Strengths, Weaknesses and Suggestions structure, plus the fact that the examples are in the repository as fixtures you can compare against.

Against a human reviewer, the difference is obvious and worth stating plainly. This is not a substitute for a reviewer, and the Relative Rank disclaimer says so in the project's own words. What it does offer is availability and coverage: a second pass on a draft before submission, in the same session as the writing.

The limits are structural. The core of the project is a hosted service, so the online path needs a network connection and depends on someone else's deployment; the Skill route is the offline option and it shifts the model dependency onto your own agent. The prompt injection detection is a separate check you have to remember to run, and it is a detection rather than a guarantee. The ranking mode is bounded by the 83.8% pairwise accuracy on 100 pools, and by the fact that the comparison set is what you choose or what a venue's accepted list contains. And the project itself says it is meant to help improve paper quality and acceptance probability, which is a claim about a process rather than a result.

For the licence, MIT covers the repository contents, the prompt templates and the Skill definition. The hosted service is a separate thing with its own terms, and the privacy policy and terms linked from the site are where that is set out.

The one thing to do first, whatever else, is to run the prompt injection check on any PDF you did not write yourself, because the whole review path depends on a model reading untrusted text.

Editorial conclusion

Ai-Review suits a researcher who wants a second opinion on a manuscript before submission, since the prompt templates in Prompts/ and the example reviews in Examples/ are the substance of the project and the web app needs nothing but a browser. It is the wrong tool if you are looking for a decision on your paper, because the README is explicit that Relative Rank is a relative ranking among the papers in the pool and not a conference decision or an accept or reject prediction. Before relying on it, run the prompt injection check on any PDF you did not write yourself, and read Examples/Relative_Rank_evaluation.md to see the measured behaviour of the ranking mode.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. NeuroDong/Ai-Review on GitHub
4. README