anc95/ChatGPT-CodeReview: a Probot bot that comments on pull requests

The problem cr-gpt solves, and the team it fits

Reviewing a pull request is mostly reading a diff and writing the same class of comment again and again: this branch is unreachable, this variable is shadowed, this error is swallowed. anc95/ChatGPT-CodeReview exists to put a model between the diff and the reviewer so that a first pass of comments appears in the pull request timeline before a human opens it. The README describes it as "A code review robot powered by ChatGPT."

The audience is narrow and specific. You need a GitHub repository, a model endpoint that speaks the OpenAI chat completions shape (OpenAI itself, Azure, or GitHub Models), and a willingness to let an external service read your diff. Teams that already run Probot apps will recognise the shape immediately; teams that do not have a JavaScript runtime in their CI story will find the GitHub Action path easier. The project is not a linter and does not run your tests. It reads a patch and writes prose about it.

How a pull request becomes a review comment

The repository layout tells most of the story. package.json depends on probot, @probot/adapter-github-actions and @probot/adapter-aws-lambda-serverless, with openai as the model client. That is one codebase compiled into three deployment targets: a long-running Probot server, a GitHub Action bundle produced by ncc into action/, and a Lambda bundle produced by build:lambda. The same review logic sits behind all three.

The trigger is the pull_request event with types opened, reopened and synchronize, as shown in the workflow example. On that event the bot fetches the patch, filters files through IGNORE_PATTERNS and INCLUDE_PATTERNS (both use minimatch, which is why the README accepts glob or regex style patterns), and sends the remaining diff to the model with the configured PROMPT, temperature, top_p and max_tokens. The reply is posted back to the pull request. On a subsequent git push the synchronize event fires again and the changed files are re-reviewed.

One design decision is worth calling out because it is a real constraint rather than a detail. MAX_PATCH_LENGTH is a hard cutoff: the README states that a patch longer than that value "will be ignored and won't review." A large generated file or a lockfile diff can therefore silently remove a pull request from review unless IGNORE_PATTERNS excludes it. The default, when MAX_PATCH_LENGTH is unset, is no limit at all, which pushes the cost and context-window problem onto the model endpoint instead.

Installing the GitHub App or the Action

There are three install paths in the README, and they are not equivalent. The hosted app at apps/cr-gpt is the fastest, but the README is explicit that it is a test deployment: "Due to cost considerations, BOT is only used for testing purposes and is currently deployed on AWS Lambda with ratelimit restrictions." Treat that as a demo, not as infrastructure.

The GitHub Action is the middle path. Add OPENAI_API_KEY to your repository secrets, then create a workflow file. This is the example from the README, trimmed to the standard OpenAI branch:

name: Code Review

permissions:
  contents: read
  pull-requests: write

on:
  pull_request:
    types: [opened, reopened, synchronize]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: anc95/ChatGPT-CodeReview@main
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          OPENAI_API_ENDPOINT: https://api.openai.com/v1
          MODEL: gpt-3.5-turbo
          LANGUAGE: Chinese

After the workflow runs on a new pull request, the review text appears in the pull request timeline and on the file changes view. The README links pull request 21 in the project's own repository as a worked example.

Self-hosting is the third path and the one the README recommends. Clone the repository, copy .env.example to .env and fill it in, then build and run under pm2:

npm i
npm i -g pm2
npm run build
pm2 start pm2.config.cjs

The .env.example file is the real configuration reference. It lists OPENAI_API_KEY, APP_ID, WEBHOOK_SECRET, PRIVATE_KEY, LOG_LEVEL, WEBHOOK_PROXY_URL, LANGUAGE, MODEL, temperature, top_p, max_tokens, REASONING_EFFORT, TARGET_LABEL, MAX_PATCH_LENGTH, PROMPT, IGNORE_PATTERNS and INCLUDE_PATTERNS. Note that the README's GitHub App instructions tell you to create OPENAI_API_KEY as a repository variable, while the Action instructions say to put it in secrets. Follow whichever path you actually chose; the two sections disagree.

There is also a Docker path, which builds the same image the Probot server runs:

docker build -t cr-bot .
docker run -e APP_ID=<app-id> -e PRIVATE_KEY=<pem-value> cr-bot

The Dockerfile installs with yarn --production --frozen-lockfile on node:18-slim and starts with yarn start, so the container expects the same environment variables as the pm2 path.

Where cr-gpt gets in the way

The most consequential limitation is the diff-length cutoff. Because MAX_PATCH_LENGTH causes an oversized patch to be skipped rather than truncated or summarised, a single large file in a pull request can suppress the review of that file with no comment explaining why. The README does not document any warning being posted. If your repository contains generated code, you need IGNORE_PATTERNS to carry the load, and the default in .env.example is /node_modules,*.md, which is a starting point rather than a policy.

Cost is the second constraint, and the project's own README admits it shaped the hosted deployment. Every synchronize event is a new model call over the changed files. A branch that receives ten pushes gets ten review passes. There is no batching or debounce described in the README, and no caching layer visible in the file listing.

The third is host coverage. The repository is a Probot app with GitHub-specific adapters. If you are searching for a Bitbucket or Atlassian equivalent, this project does not address that case at all: the README, action.yml and app.yml are all GitHub surfaces. Finally, the model output is prose. The bot does not run a linter, does not fail a check, and does not block a merge, so it cannot be used as a gate.

How it differs from codereview.gpt

The README credits codereview.gpt as the inspiration, and the two take different routes to the same idea. codereview.gpt is a browser-side tool: you paste a diff and get comments back, which means nothing is installed and nothing runs automatically, but also that the review is manual and never appears in a pull request.

cr-gpt moves the same model call into the GitHub event stream. The trade is automation and placement for operational weight. You now own a webhook endpoint, an APP_ID and a private key, or a workflow file and a secret. In exchange, the review lands where reviewers already are, and it re-runs on every push. For a team that reviews a handful of pull requests a week, pasting diffs might be the cheaper habit. For a team that wants the first comment to exist before anyone opens the tab, the Probot route is the point of the project.

A second difference worth noting is deployment flexibility. Because the code compiles to a GitHub Action bundle, a Lambda bundle and a long-running server from one source tree, you can start with the Action and move to self-hosting later without changing the review logic.

Maintenance, licence and upgrade cost

The repository is not archived and the last push was on 2026-08-10, roughly a month before this writing. Releases are infrequent and irregular rather than rapid: v1.0.24 on 2026-07-14, v1.0.23 on 2026-02-07, and v1.0.22 on 2025-08-04. That cadence fits a small utility whose upstream dependency, the OpenAI client, is the part most likely to move. package.json pins openai at ^6.45.0 and probot at ^12.2.4, and the engines field requires Node 18 or newer, so a self-hosted deployment inherits the usual Node runtime upgrade treadmill.

Upgrade cost is low if you use the Action, because the workflow references anc95/ChatGPT-CodeReview@main and you can pin a tag instead. It is higher if you self-host, because the build script runs rollup and then ncc, and a Probot major version bump would land in your own dependency tree rather than in someone else's workflow.

The licence is ISC, a permissive licence structurally similar to MIT. That means you can use, modify and redistribute the code, including in commercial settings, provided the copyright notice and permission notice are retained. The LICENSE file carries the copyright line "ISC © 2023 anc95." This is a description of the licence text, not legal advice; if you plan to redistribute a modified build, read the LICENSE file and your own counsel's guidance.

Editorial conclusion

Adopt it if you already pay for an OpenAI or Azure model endpoint and you want diff-level comments inside the GitHub review UI without writing the Probot plumbing yourself. Skip it if you need Bitbucket or Atlassian support, since nothing in the README or the repository layout addresses those hosts, or if you expect the hosted app at apps/cr-gpt to behave like production infrastructure. Verify two things before rolling it out broadly: how MAX_PATCH_LENGTH and IGNORE_PATTERNS interact on your largest pull requests, and whether your model endpoint accepts the temperature, top_p and max_tokens values you plan to set in .env.

Frequently asked questions

Official sources

1. anc95/ChatGPT-CodeReview on GitHub
2. License: ISC
3. Project website
4. README
5. Releases