Anubis: A Go-Based Challenge Firewall for Blocking AI Crawlers

The Problem Anubis Targets

AI companies send an endless storm of requests to scrape websites, and small site operators often lack the infrastructure to filter them. Anubis is built for that specific fight. It is a Web AI Firewall Utility that weighs the soul of your connection using one or more challenges before letting a request reach upstream resources. The README is blunt about the target: protecting the small internet from scraper bots. This is not a general-purpose bot manager. It is a defensive tool for people who are being overwhelmed by automated traffic from AI vendors and who want a self-hosted option.

How Anubis Works: Challenges as a Gate

The mechanism is a challenge-response gate. When an HTTP request arrives, Anubis presents one or more challenges that a client must solve before the request is proxied to the upstream origin. The README does not name the challenge types, but the concept is clear: the service sits in front of your origin and only forwards requests that pass the challenge. The Cargo workspace includes wasm/pow/* members, which points to proof-of-work challenges compiled to WebAssembly. The critical design point is that Anubis is lightweight by intent. The project states it is as lightweight as possible so that anyone can afford to run it. That means the challenge logic is meant to be cheap to execute, not a heavy AI classifier.

Getting It Running: Commands and Configuration

The README does not include explicit installation commands, but it points to a documentation site at anubis.techaro.lol. That site is the primary source for setup instructions. The repository is Go-based, so a typical deployment would involve building the binary or using a container image, though the README does not confirm those details. What is confirmed is that you can configure bot policy definitions, as referenced in the policies file at docs/docs/admin/policies.mdx. That file is the place to define allowlists for known good bots. The README mentions that the project is working on a curated set of known good bots, so initial configuration likely requires manual allowlisting.

The Nuclear Response: Blocking Good Bots

The most important limitation is stated directly in the README: Anubis is a bit of a nuclear response. It will block smaller scrapers and may inhibit good bots like the Internet Archive. That is a serious trade-off for any site that cares about archival or search engine discovery. The README acknowledges this and suggests that most sites should not need Anubis, and that Cloudflare is often sufficient. If your site depends on being indexed by the Internet Archive or similar services, you will need to explicitly allowlist them using the bot policy definitions. The project is working on a curated allowlist, but as of the latest release, that is not guaranteed to be complete. This is not a tool for sites that want to remain open to all traffic; it is a gate that requires deliberate maintenance.

Comparison with Cloudflare

The README makes a direct comparison: in most cases, you should not need this and can probably get by using Cloudflare to protect a given origin. Cloudflare offers bot management as a managed service, with global infrastructure and a dashboard. Anubis is self-hosted, open source, and runs on your own hardware. The difference in approach is stark: Cloudflare is a proxy that filters traffic at its edge, while Anubis is a reverse proxy that you run yourself. If you are already behind Cloudflare, the README suggests you can probably get by without Anubis. If you cannot or will not use Cloudflare, Anubis is the alternative. That is the core choice. Anubis gives you control, but it also gives you the operational burden of running and maintaining another service.

Licence, Maintenance, and Upgrade Cost

Anubis is released under the MIT license, which is permissive and allows commercial use with attribution. The project is actively maintained, with a recent release v1.28.2 on 2026-10-23, following v1.28.1 and v1.28.0 earlier in October 2026. The maintenance cost is not documented in the README, but the need to maintain a bot allowlist is a clear ongoing requirement. Upgrade cost is minimal if you follow the release notes, but the project does not provide a migration guide in the README. You should expect to track the documentation site for changes to policy configuration. The sponsor list includes companies like Gitea and Weblate, which suggests a community-backed project, but that does not guarantee long-term support.

Editorial conclusion

Adopt Anubis if you run a small site that is being hammered by AI crawlers and you cannot or will not use Cloudflare, and you accept that it will also block smaller scrapers and may inhibit good bots like the Internet Archive. Do not adopt it if your site relies on broad discoverability or if you have no way to maintain an allowlist of known good bots. Before deploying, verify that your upstream services can tolerate the extra latency and that you can configure the bot policy definitions to allowlist the bots you care about. Anubis is a deliberate trade-off; it is not a general-purpose rate limiter.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes