Anubis: A Go-Based Challenge Firewall for Blocking AI Crawlers
Weighs the soul of incoming HTTP requests to stop AI crawlers
At a glance
- What is it?
- Anubis is a lightweight Go service that challenges incoming HTTP requests to protect small websites from AI scraper bots. It is a nuclear option that blocks many good bots, so it suits sites that cannot or will not use Cloudflare.
- Who is it for?
- Adopt Anubis if you run a small site that is being hammered by AI crawlers and you cannot or will not use Cloudflare, and you accept that it will also block smaller scrapers and many good bots like the Internet Archive. Do not adopt it if your site relies on broad discoverability or if you have no way to maintain an allowlist of known good bots.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly Go, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Problem Anubis Targets
AI companies send an endless storm of requests to scrape websites, and small site operators often lack the infrastructure to filter them. Anubis is built for that specific fight. It is a Web AI Firewall Utility that weighs the soul of your connection using one or more challenges before letting a request reach upstream resources. The README is blunt about the target: protecting the small internet from scraper bots. This is not a general-purpose bot manager. It is a defensive tool for people who are being overwhelmed by automated traffic from AI vendors and who want a self-hosted option.
How Anubis Works: Challenges as a Gate
The mechanism is a challenge-response gate. When an HTTP request arrives, Anubis presents one or more challenges that a client must solve before the request is proxied to the upstream origin. The documentation does not specify the exact challenge types, but the concept is clear: the service sits in front of your origin and only forwards requests that pass the challenge. This is similar to a proof-of-work or CAPTCHA-style gate, though the README does not confirm which. The critical design point is that Anubis is lightweight by intent. The project states it is as lightweight as possible so that anyone can afford to run it. That means the challenge logic is meant to be cheap to execute, not a heavy AI classifier.
Getting It Running: Commands and Configuration
The README does not include explicit installation commands, but it points to a documentation site at anubis.techaro.lol. That site is the primary source for setup instructions. The repository is Go-based, so a typical deployment would involve building the binary or using a container image, though the README does not confirm those details. What is confirmed is that you can configure bot policy definitions, as referenced in the policies file at docs/docs/admin/policies.mdx. That file is the place to define allowlists for known good bots. The README mentions that the project is working on a curated set of known good bots, so initial configuration likely requires manual allowlisting. For a quick test, you can visit the live demo site, which is mentioned as a way to try it out.
The Nuclear Response: Blocking Good Bots
The most important limitation is stated directly in the README: Anubis is a bit of a nuclear response. It will block smaller scrapers and may inhibit good bots like the Internet Archive. That is a serious trade-off for any site that cares about archival or search engine discovery. The README acknowledges this and suggests that most sites should not need Anubis, and that Cloudflare is often sufficient. If your site depends on being indexed by the Internet Archive or similar services, you will need to explicitly allowlist them using the bot policy definitions. The project is working on a curated allowlist, but as of the latest release, that is not guaranteed to be complete. This is not a tool for sites that want to remain open to all traffic; it is a gate that requires deliberate maintenance.
Comparison with Cloudflare
The README makes a direct comparison: in most cases, you should not need this and can probably get by using Cloudflare to protect a given origin. Cloudflare offers bot management as a managed service, with global infrastructure and a dashboard. Anubis is self-hosted, open source, and runs on your own hardware. The difference in approach is stark: Cloudflare is a proxy that filters traffic at its edge, while Anubis is a reverse proxy that you run yourself. If you are already behind Cloudflare, Anubis is redundant. If you cannot or will not use Cloudflare, Anubis is the alternative. That is the core choice. Anubis gives you control, but it also gives you the operational burden of running and maintaining another service.
Licence, Maintenance, and Upgrade Cost
Anubis is released under the MIT license, which is permissive and allows commercial use with attribution. The project is actively maintained, with a recent release v1.27.0 on 2026-08-08. The release naming includes pre-releases, indicating a regular release cadence. The maintenance cost is not documented in the README, but the need to maintain a bot allowlist is a clear ongoing requirement. Upgrade cost is minimal if you follow the release notes, but the project does not provide a migration guide in the README. You should expect to track the documentation site for changes to policy configuration. The sponsor list includes companies like Gitea and Weblate, which suggests a community-backed project, but that does not guarantee long-term support.
Editorial conclusion
Adopt Anubis if you run a small site that is being hammered by AI crawlers and you cannot or will not use Cloudflare, and you accept that it will also block smaller scrapers and many good bots like the Internet Archive. Do not adopt it if your site relies on broad discoverability or if you have no way to maintain an allowlist of known good bots. Before deploying, verify that your upstream services can tolerate the extra latency and that you can configure the bot policy definitions to allowlist the bots you care about. Anubis is a deliberate trade-off; it is not a general-purpose rate limiter.
Community notes