Krawl: a self-hosted web deception server that feeds attackers fake vulnerable apps

The problem Krawl addresses: scanners are cheap, and your logs are noisy

Automated scanners and crawlers hit every public hostname. Separating a credential-stuffing run from a legitimate search engine crawl in ordinary access logs is tedious, and the log line usually arrives after the request has already been served. Krawl takes the opposite approach. Instead of trying to filter real traffic, it publishes a decoy application that nothing legitimate should ever touch. The README describes it as a deception server that creates "realistic fake web applications filled with low-hanging fruit such as admin panels, configuration files, and exposed fake credentials." Anything that interacts with those pages is, by construction, not a normal user. That is the whole detection model: a request to a fake phpMyAdmin login is a stronger signal than a suspicious User-Agent string, because no honest client has a reason to be there. The audience is blue teams, homelab operators, and anyone running a public-facing service who wants an early-warning tripwire rather than another log filter.

What the deception surface actually contains

The README lists the components: fake login pages for WordPress, phpMyAdmin and generic admin panels; honeypot paths advertised in robots.txt; realistic-looking usernames, passwords and API keys; random server headers and version strings; random error injection to mimic real server behaviour; and spider trap pages that generate infinite random links, based on the spidertrap project. Canary token integration is documented separately and is described as triggering external alerts. There is also an AI generation path documented in docs/ai_generation.md, which the README frames as letting attackers help generate the fake vulnerable attack surface. The demo instance at demo.krawlme.com is public, and the README suggests crawling its robots.txt paths. Note what is not claimed: there is no statement that Krawl blocks or mitigates anything by itself. It detects and records. The banlist is a separate artefact, published weekly in banlist.txt, containing IPs from attackers that triggered honeypot traps, and it is downloadable without running Krawl at all.

Standalone versus scalable: two modes, two dependency sets

The mode setting in config.yaml, or the KRAWL_MODE environment variable, selects between two architectures. Standalone uses SQLite in WAL mode for storage and an in-memory Python dictionary for cache, runs as a single replica, and needs no external dependencies. The README positions it for development, homelabs, and under 500k requests, with lower RAM use. Scalable swaps SQLite for PostgreSQL and the in-memory dict for Redis with multi-tier TTL caching, supports one or more replicas behind a load balancer, and is described as giving a more responsive dashboard and lower disk I/O because Redis sits in front of PostgreSQL. The Helm chart defaults to scalable mode, which is worth knowing before you install it into a small cluster and wonder why PostgreSQL and Redis are being pulled in. Migration instructions between the two are in docs/deployment-modes.md. The request-count figure is the project's own guidance, not a measured threshold, so treat it as a rough boundary rather than a capacity plan.

Getting it running: four documented paths

The Quickstart section lists Docker run, Docker Compose, Kubernetes, and Uvicorn for a direct Python run. Images are published at ghcr.io/blessedrebus/krawl, and a Helm chart is published separately as krawl-chart. Configuration comes from config.yaml or from environment variables. Two keys matter immediately. KRAWL_MODE selects standalone or scalable. KRAWL_DASHBOARD_SECRET_PATH sets the dashboard path; if you do not set it, Krawl generates a random secret path at startup, which keeps the dashboard hidden from scanners but also means you have to find the generated value before you can log in. The dashboard lives at /das_dashboard in the demo, and the banlist export endpoint is documented as /das_dashboard/api/export-ips?categories=. Two protected tabs, Tracked IPs and IP Banlist, appear only after authenticating with the dashboard password, and a Deception tab lets you import and export AI-generated pages. If you run Krawl behind a reverse proxy or CDN, the README has a dedicated section for that, and it is not optional reading: without correct forwarding configuration the recorded client IP will be your proxy, and every entry in the dashboard and the banlist export will be wrong.

Where Krawl is the wrong tool

A deception server only works if it is deployed somewhere attackers can reach and legitimate users cannot. If you have a single hostname serving real traffic, adding Krawl to it means your real users will eventually hit a fake login page and file a support ticket. The project's own diagram shows Krawl exposed alongside other services through a reverse proxy, which implies you need a spare hostname or path namespace and the proxy configuration to route it. Second, Krawl records activity; it does not stop it. If your goal is blocking, you are looking at the banlist export and an external enforcement step, and the README's IPTables export format is the documented handoff. Third, the scalable mode is not free: PostgreSQL and Redis are two more stateful services to run, back up, and upgrade. For a homelab, standalone is the sane default; for a production deployment where the dashboard is expected to stay responsive under load, you are signing up for the full dependency set. Fourth, the AI-generated page feature depends on an external generation path documented in docs/ai_generation.md, and the README does not describe what happens when that path is unavailable.

Compared with a conventional honeypot such as Cowrie

Cowrie is the obvious reference point for anyone evaluating Krawl, and the difference is the attack surface. Cowrie emulates SSH and Telnet, so it captures interactive shell sessions, brute-force attempts, and post-authentication commands. It is a protocol-level trap. Krawl is an HTTP-level trap: it serves fake web applications, fake admin panels, fake credentials, and spider traps, and it reports on crawlers, scanners, and credential harvesters that operate over HTTP. The two do not overlap much. If your exposure is a public web hostname being scanned by automated tooling, Krawl addresses that directly and Cowrie does not. If your exposure is an SSH port being brute-forced, the reverse is true. A team running both is not duplicating effort, but it is running two services with two dashboards and two sets of logs, and Krawl's dashboard is the only place its data lands unless you export it.

Maintenance, releases, and the MIT licence

Krawl is MIT licensed, which permits commercial use, modification, and redistribution provided the copyright notice and permission notice are retained. That is a permissive arrangement and it matters here: the banlist, the wordlists, and the AI-generated page templates are all things you may want to adapt, and MIT does not restrict that. It is not legal advice, and if you redistribute Krawl inside a product you should read the actual LICENSE file in the repository rather than this summary. On maintenance, the release history shows v2.2.0 in June 2026, v2.3.0 in August 2026, and v2.3.1 later that same month, with the last push to main in September 2026. That is a project that is being actively patched, and the minor-version cadence suggests features are still arriving. The upgrade cost is concentrated in the mode boundary: standalone deployments upgrade by pulling a new image, while scalable deployments have to consider PostgreSQL schema changes and Redis cache invalidation on each release. The README does not document a migration tool for schema changes, so check the release notes for each version before upgrading a scalable deployment.

Editorial conclusion

Adopt Krawl if you already run a reverse proxy or a Kubernetes cluster and want a deception surface you can point scanners at without touching production data. Do not adopt it if you have no spare hostname, no ability to route traffic to a separate backend, or no one to read the dashboard. Before deploying, verify three things: that the mode setting in config.yaml matches your database and cache (SQLite and in-memory dict for standalone, PostgreSQL and Redis for scalable), that KRAWL_DASHBOARD_SECRET_PATH is set to a value you control rather than left to the startup-generated random path, and that your reverse proxy or CDN is configured so the recorded client IP is the attacker's and not the proxy's.

Official sources

1. BlessedRebuS/Krawl on GitHub
2. License: MIT
3. Project website
4. README
5. Releases