CrawlSEO: a self-hosted SEO dashboard that puts GSC, a crawler and vitals in one Postgres

The problem CrawlSEO targets: SEO data spread across four paid tabs

A small team tracking organic search usually ends up with a browser full of tabs. Search Console shows queries and impressions but no crawl detail. A crawler tool shows broken links and thin content but not clicks. PageSpeed Insights shows LCP and CLS but nothing about which pages earn traffic. Backlink and keyword tools sit behind separate subscriptions. CrawlSEO's README states the goal plainly in its subtitle: open-source SEO monitoring for founders, not SEO specialists. The comparison table in the repository pitches the project at $0 against OpenSEO at $10/mo, Ahrefs at 119 euros/mo, Semrush at $139/mo and Moz at $49/mo, and marks itself as self-hosted and MIT licensed. That framing tells you the intended user: someone running a site who wants clicks, impressions, positions, crawl issues and vitals in one place, and who is willing to run a container to get it. It is not aimed at an agency managing dozens of client properties, because nothing in the material describes multi-tenant workflows, client reporting or seat management.

What actually runs: Next.js, Prisma, Postgres and a separate MCP process

The stack table lists Next.js 16 with the App Router, TypeScript, PostgreSQL, Prisma as the ORM, NextAuth.js v5 for authentication, shadcn/ui with Tailwind CSS v4 for the interface, Recharts for charts and Lucide React for icons. The MCP layer uses @modelcontextprotocol/sdk and is invoked as a separate process rather than embedded in the web app: the README's Claude Code configuration runs npx tsx mcp/server.ts with a cwd pointing at the repository. That separation matters. The dashboard and the agent interface are two entry points onto the same database, so an agent calling get_keywords or run_crawl is reading and writing the same Postgres rows the web UI renders. Data flows in from three external sources. Google Search Console supplies keywords, pages, clicks, impressions and position data with 28-day comparison and delta indicators. The crawler fetches your own site concurrently, up to 2,000 pages, and produces a health score, 16 issue types, content scoring and remediation guidance. PageSpeed Insights supplies LCP, CLS, INP and TTFB with mobile and desktop comparison. Alerts are described as firing on traffic drops, position changes, new 404s and vitals degradation, delivered over email, Slack, Telegram or webhook.

Getting it running: two paths, one OAuth detour

The development path in the README is a short sequence: git clone the repository, cd into it, cp .env.example .env.local, add Google OAuth credentials, then docker compose up -d db to start only the database, npm install, npx prisma migrate dev --name init, and npm run dev. The app then answers on http://localhost:3000. The OAuth setup is the part that will absorb your time. You create or select a Google Cloud project, enable the Google Search Console API, create an OAuth 2.0 Client ID of type Web application, and register http://localhost:3000/api/auth/callback/google as an authorized redirect URI. The required scopes are listed as openid, email, profile and https://www.googleapis.com/auth/webmasters.readonly. Note the readonly scope: the material does not describe any write path back into Search Console, so treat this as a read-only integration. The self-hosting path is shorter: cp .env.example .env, edit it, then docker compose pull and docker compose up -d. Compose pulls ghcr.io/crawlseo/crawlseo:latest, images support linux/amd64 and linux/arm64, and migrations run automatically when the container starts. To pin a release or use a fork, set CRAWLSEO_IMAGE in .env, for example CRAWLSEO_IMAGE=ghcr.io/crawlseo/crawlseo:1.2.3. To build locally instead, run docker build -t crawlseo:local . and then CRAWLSEO_IMAGE=crawlseo:local docker compose up -d.

The MCP server is the most differentiated piece, and the least documented here

Ten tools are exposed, grouped in the README as Sites (list_sites, get_site_overview), Keywords and Pages (get_keywords, get_pages, get_traffic), Crawl and Audit (run_crawl, get_crawl_status, get_crawl_issues) and Performance (get_vitals, get_opportunities). The split between run_crawl and get_crawl_status implies crawls are asynchronous: an agent starts a job and polls rather than blocking on a 2,000-page fetch. That is the right shape for an agent tool, and it is also a design constraint worth understanding, because it means an agent session that calls run_crawl and immediately reads results will get nothing useful. The README points to mcp/README.md for the full setup guide and names Claude Code, Claude Desktop and Cursor as supported clients. What the supplied material does not cover is authentication and scoping for the MCP process itself. The web app uses NextAuth with Google, but the MCP entry point is a local tsx process pointed at the repository directory. Whether it reads the same session, a service token, or simply trusts filesystem access is not stated. If you intend to expose this beyond a developer laptop, that gap is the first thing to resolve.

Where the free tier stops: BYOK keyword and backlink data

The feature table marks keyword research and backlinks as BYOK, defined in the README as Bring Your Own Key. Keyword research uses DataForSEO for ideas with volume, difficulty and CPC, and backlink analysis covers referring domains, anchor text and dofollow/nofollow classification. Google Autocomplete is offered as a free fallback for keyword suggestions. This is the clearest boundary in the project. Everything that reads your own Search Console property, crawls your own pages and measures your own vitals is self-contained. Everything that requires a third party's index of the wider web routes through DataForSEO and its pricing. The README calls DataForSEO optional, which is accurate in the sense that the dashboard runs without it, but the keyword and backlink screens will be empty or Autocomplete-only if you skip it. If backlink data is the reason you are evaluating CrawlSEO, you are comparing a DataForSEO bill plus your own hosting against an Ahrefs or Semrush subscription, not comparing free against paid.

The case against: 2,000 pages, one release, and a maintenance surface

CrawlSEO is the wrong tool in a few concrete situations. First, size. The crawler is documented at up to 2,000 pages. A site with tens of thousands of URLs, or one where you need crawl scheduling across segments, will hit that ceiling and there is no described mechanism for sharding a crawl across runs. Second, maturity. The repository shows a single release, v0.1.0, published 2026-08-05, with the last push on 2026-09-05. A 0.1.0 project with a four-week gap between release and last commit is early. The README's own logo markup is still a commented-out placeholder pointing at docs/screenshots/logo.png, which is a small but telling signal about how much polish work remains. Third, operating cost. You are running Next.js, Postgres and a Prisma migration pipeline. The Docker path automates migrations on container start, which is convenient and also means a bad migration ships the moment you pull. Anyone who has not operated a Postgres-backed web app before should weigh that against the subscription they are avoiding. Fourth, the material does not describe rate limiting, retry behaviour against the Search Console API, or how the crawler handles robots.txt and crawl-delay directives. Those are the questions that decide whether a crawler is safe to point at a production site, and they are unanswered here.

Compared with OpenSEO, the closest thing in the same category

The README's own table places OpenSEO at $10/mo, self-hosted, open source, with GSC integration, a site crawler, keyword research and backlinks, and 24 MCP tools against CrawlSEO's 10. The rows where they differ are Core Web Vitals, where CrawlSEO claims support and OpenSEO does not, and AI agent readiness, where both claim support. So the honest comparison is not free versus paid. It is a paid hosted-or-self-hosted tool with a broader MCP surface and a commercial backer, against an MIT project with a narrower tool set that adds PageSpeed-derived vitals. If your workflow is agent-driven and you need many MCP tools, 24 beats 10 on the face of it. If you specifically want LCP, CLS, INP and TTFB sitting next to your crawl issues and your Search Console clicks, CrawlSEO is the one that claims to do that in a single dashboard. The other comparison worth naming is the DIY route: a scheduled Search Console export into a warehouse plus a standalone crawler. That gives you full control of the schema and no application to upgrade, at the cost of building every view yourself.

Licence, upgrades and what to check before you commit

The licence is MIT, stated in the README badge and in the repository metadata. MIT permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is permissive, and it is worth noting because a company that later wants to host CrawlSEO for clients is not obviously blocked by the licence. This is not legal advice; read the LICENSE file and, if you plan to redistribute or host it commercially, get your own review. On upgrades, the material supports a specific workflow: version tags are published as immutable image tags such as 1.2.3 and as minor-version tags such as 1.2, and CRAWLSEO_IMAGE in .env selects one. Pinning to a full version tag rather than latest is the pattern the README itself describes, and it is the one that keeps an unattended docker compose pull from moving your database schema. The automatic migration on container start is the reason to care. Verify before adoption: your OAuth redirect URI and scopes, whether the 2,000-page crawl limit covers your site, whether DataForSEO is in your budget for keyword and backlink screens, and whether mcp/README.md answers the authentication question the top-level README leaves open.

Editorial conclusion

Adopt CrawlSEO if you run your own infrastructure, already have a Google Cloud project for Search Console, and want your crawl and vitals data in a Postgres you control rather than in a vendor's dashboard. Do not adopt it if you need backlink or keyword data without supplying a DataForSEO key, or if nobody on the team will maintain a Node 20 and Postgres deployment. Before committing, verify three things on your own machine: that the OAuth redirect URI http://localhost:3000/api/auth/callback/google matches what you registered, that the webmasters.readonly scope is enough for the reports you need, and that the 2,000-page crawl ceiling covers your site.

Official sources

1. crawlseo/crawlseo on GitHub
2. Issues
3. License: MIT
4. README
5. Releases