AI-Codereview-Gitlab: LLM Review on GitLab Webhooks, With a Tool-Calling Mode You Pay For in Tokens

The gap it fills: review comments that arrive without anyone opening the diff

Most teams do not lack a linter. They lack a reviewer who reads the merge request before the author has already merged it. This project targets that specific window. GitLab fires a webhook on a Push or Merge Request event, the service calls a large language model with the diff, and the model's output is written back as a Note on the merge request or commit. The README's own summary of the mechanism is short: GitLab triggers a webhook, the system reviews the code through a third-party model, and the result is fed back into the corresponding Note.

Who it is for is fairly narrow and the README does not pretend otherwise. You need GitLab, because the webhook contract is GitLab's. You need a model endpoint you are willing to pay for, whether that is DeepSeek, ZhipuAI, OpenAI, Anthropic, Qwen or a local Ollama instance. And you need the service reachable from GitLab, which the README flags directly: if your internal network is restricted, it suggests deploying on an external server. That single sentence rules out a lot of locked-down corporate environments, and it is worth reading as a design constraint rather than a footnote.

The data flow: webhook in, diff out, note back, optionally a cloned repo in the middle

There are two distinct pipelines behind the same webhook endpoint, selected by the REVIEW_STRATEGY environment variable.

The default, diff_only, is the simpler one and the README states it behaves identically to the original version. The service receives the event, extracts the diff, filters files by SUPPORTED_EXTENSIONS, sends the text to the configured LLM_PROVIDER, and posts the returned text back to GitLab as a Note. Nothing is cloned. The model sees only what changed, with no surrounding context.

The agentic strategy changes the shape of the system. The service clones or fetches the target repository into REPO_CACHE_DIR, then hands the model a tool interface with read_file and a sandboxed run_command. The model can walk the codebase on its own, up to AGENT_MAX_ITERATIONS rounds, before producing a review. The README describes this as producing a more comprehensive result, which is a fair description of the mechanism but not a measured claim.

What makes the agentic path defensible rather than reckless is the fallback. The README states that failure at any stage (clone, fetch, LLM call, tool invocation) degrades automatically back to diff_only, so the worst case is a review equivalent to the default. That is the right failure direction for a webhook handler, where a thrown exception means the author gets nothing.

The shell tool is constrained by a command allowlist, a blocklist, path traversal checks and a 30 second timeout. By default only read-oriented commands are permitted, and the README names ls, cat, grep, find and git log as examples. Widening that is done through AGENT_SHELL_ALLOWLIST and AGENT_SHELL_BLOCKLIST. Note the framing in the README: the sandbox is described as a defence, and the defaults are conservative, but the allowlist is an environment variable. Anyone who can set environment variables on the host can widen the command surface.

Getting it running: two deployment paths and the .env keys that actually matter

The Docker path is the shorter one. Clone the repository, copy the sample environment file, edit it, and bring the stack up:

git clone https://github.com/sunmh207/AI-Codereview-Gitlab.git cd AI-Codereview-Gitlab cp conf/.env.dist conf/.env docker-compose up -d

The keys you cannot skip are LLM_PROVIDER, the matching API key for that provider (the README shows DEEPSEEK_API_KEY as the example), GITLAB_ACCESS_TOKEN, and SUPPORTED_EXTENSIONS. That last one is easy to overlook and it is a hard filter: the README says explicitly that file types not configured will not be reviewed. The sample value is a comma-separated list of extensions including .java, .py, .php, .yml, .vue, .go, .c, .cpp, .h, .js, .css, .md and .sql. If your team writes Rust or Kotlin, nothing gets reviewed until you add those extensions.

Verification is two ports. Port 5001 serves the main service and should return the string "The code review server is running." Port 5002 serves the Streamlit dashboard, which the README describes as a review log page.

On the GitLab side, create either a Personal Access Token or a Project Access Token, then add a webhook pointing at http://{your-server-ip}:5001/review/webhook with Push Events and Merge Request Events checked. The README warns against checking other events. Token resolution has a stated priority order: GITLAB_ACCESS_TOKEN from the .env file wins, and the webhook Secret Token is used only when that variable is absent.

Without Docker, the manual path is pip install -r requirements.txt on Python 3.10 or newer, then python api.py for the API and streamlit run ui.py --server.port=5002 --server.address=0.0.0.0 for the dashboard. Two processes, two ports, same .env file.

The agentic mode's real bill: tokens, disk and a thirty-second-to-five-minute wait

The README publishes the overhead numbers for agentic review, and they are the most useful part of the document. Disk: reserve at least 50GB, because repositories are cloned into REPO_CACHE_DIR and the README estimates 10MB to 2GB per project. Memory: roughly 500MB peak per session. Tokens: 5,000 to 50,000 per review, which the README puts at 3 to 10 times the diff_only cost. Latency: 30 seconds to 5 minutes per review.

Those figures reframe the feature. A merge request review that takes five minutes is not a gate you can put in front of a fast-moving trunk. It is a background comment. And a 10x token multiplier on a busy repository turns a cheap utility into a line item. The REPO_CACHE_DIR default is data/repo_cache/, which on a Docker deployment means a volume you need to plan for rather than a directory that grows quietly on the host.

There is a second cost the README does not quantify: freshness. The cache holds a cloned copy, and the review quality depends on that copy being current with the merge request under review. The README says the service clones or updates as needed and that a fetch failure degrades to diff_only, which is a reasonable safety net, but it also means a fetch problem silently downgrades your review quality rather than raising an error you would notice.

My read: turn agentic mode on for a small number of high-risk repositories, not as a default across an organisation, and watch the token counter before you widen it.

Where it will disappoint you: no context, no test execution, and a tone setting that is not a review policy

The default strategy reviews a diff in isolation. The model does not see the file the change lives in, the callers of the function being modified, or the tests that cover it. For a one-line fix in a well-named function that is fine. For a change that moves a boundary between two modules, the review will be shallow, and it will sound confident anyway. That is the honest limit of diff_only, and the agentic mode exists precisely because the maintainer knows it.

Agentic mode does not close the gap completely. The shell tool is read-only by default. The model can read files and run git log, but it cannot run the test suite, so it cannot tell you that your change breaks a test. It reasons about code, it does not execute your project's checks.

The review style feature deserves a sceptical look. The README offers four personas: a professional style, a sarcastic one, a gentlemanly one and a humorous one, with sample lines like a comparison to a bad blind date. Teams that enable the sarcastic style and post the output as a merge request Note are putting that text in front of the author by name. The README presents this as a feature. It is a configuration choice with social consequences, and there is no documented filter between the model's output and the Note.

Finally, there is a Pro version. The README links to a separate document and an install script fetched over curl from the repository's main branch. The open source edition under Apache-2.0 is what this article covers; the boundary between the two is defined only in that other document, which is worth reading before you assume a feature you want is in the free tier.

What you would use instead, and how the approach differs

The obvious alternative is running review entirely inside GitLab CI: a job in .gitlab-ci.yml that calls an LLM API with the diff and posts a comment, or a self-hosted runner invoking a linter and a static analyser such as SonarQube or Semgrep. The difference is not the model. It is where the trigger lives and who owns the plumbing.

A CI job runs on the runner, uses the pipeline's own credentials, and its output is visible in the job log alongside every other check. It fails the pipeline if you want it to. It scales with your runner fleet and you already have monitoring for it. What it does not give you is the webhook-driven Note on the merge request, the dashboard on port 5002, or the scheduled daily digest assembled from GitLab, GitHub and Gitea commit records. Those are the parts of this project you cannot get from a CI snippet.

Against a hosted review service, the trade is inverted. A hosted service keeps the operational burden off your team but sends your diffs to a third party. This project sends your diffs to whichever model provider you configured, which is either the same exposure or a strictly smaller one if you point LLM_PROVIDER at a local Ollama instance. That local-model path is the strongest argument for self-hosting here, and it is the one configuration where the cost model changes completely: no per-token billing, only the hardware.

A pure static analyser is the more honest comparison for the diff_only mode. Semgrep or a language-specific linter will find a null dereference deterministically and never invent a finding. An LLM review will catch intent-level problems a linter cannot express, and will also occasionally produce a comment that is simply wrong. If your team cannot absorb the second half of that sentence, the linter is the better tool and this project is not.

Upgrades, maintenance and the licence you are actually accepting

The project is active: the most recent release listed is v1.5.1 from June 2026, with v1.4.3 and v1.4.2 before it, and the last push to main is later still. Releases arrive on a rough two-to-six week cadence based on those three dates, which is frequent enough that you should expect to pull updates rather than pin once and forget.

The upgrade surface is small and mostly declarative. The .env file is the contract: LLM_PROVIDER, the provider API key, GITLAB_ACCESS_TOKEN, SUPPORTED_EXTENSIONS, DINGTALK_ENABLED and DINGTALK_WEBHOOK_URL, plus the agentic keys REVIEW_STRATEGY, REPO_CACHE_DIR, AGENT_MAX_ITERATIONS, AGENT_SHELL_ALLOWLIST and AGENT_SHELL_BLOCKLIST. New features that need configuration will most likely appear as new keys here, which means a diff of conf/.env.dist against your conf/.env is the cheapest upgrade check you can run before restarting the container. The README does not publish a migration guide or a changelog beyond the release list, so that diff is your only reliable signal.

Operationally the persistent state is REPO_CACHE_DIR, which the README defaults to data/repo_cache/. Back it up or do not, but know that it is the one directory that grows and the one you would need to clear if a clone goes bad.

On licensing: the repository is Apache-2.0, which permits commercial use and modification and includes a patent grant, with the usual obligations around preserving notices and stating changes. The Pro version referenced in the README is a separate offering and its terms are not stated in the material available here. Nothing in this article is legal advice; if you are embedding this in a product you ship, have someone read the licence file and the Pro document rather than relying on a summary.

Editorial conclusion

Adopt it if you run self-hosted GitLab, already pay for a DeepSeek, ZhipuAI, OpenAI, Anthropic, Qwen or Ollama endpoint, and want review notes posted back into merge requests without sending code to a vendor-hosted review SaaS. Skip it if your repositories are huge, your LLM budget is metered tightly, or you cannot give the host roughly 50GB of disk and outbound network access to the model provider. Before rolling it out to a whole group, verify three things on one throwaway project: that your SUPPORTED_EXTENSIONS list covers the languages you actually merge, that a diff_only review lands as a Note on a real merge request, and that your first agentic run stays inside the token and latency envelope you are willing to pay for.

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. sunmh207/AI-Codereview-Gitlab on GitHub