Unbrowse: a local route layer that turns browsing into reusable API calls

The problem Unbrowse picks, and who has it

Most agent frameworks treat a website as something to be rendered. The agent opens a browser, waits for the DOM, dumps a page into the context window, and asks a model to read it. That works, and it costs a browser session plus a large token bill on every single call, even when the same site was visited an hour ago.

Unbrowse starts from a narrower observation: if a site already exposes a first-party route behind its own UI, the agent should reuse that route instead of rediscovering it. The README states the working claim exactly that way, and the narrowness is the point. This is a tool for agents that call the same handful of sites over and over, not for one-off research where nothing is worth caching.

The intended audience is visible in the repository layout. There are configuration directories for a long list of coding agents (.codex/, .goose/, .qwen/, .roo/, .junie/), a SKILL.md at the top level, an .mcp.json, and workspace packages named playwright-shim, firecrawl-shim and stagehand-shim. Unbrowse is built to sit underneath existing agent tooling rather than replace it.

How the route graph and the hole contract fit together

The public interface is a contract with five fillable holes: intent, wallet_proof, approval, local_capability_result and typed_pointer. The model fills only what it can know. The runtime decides which layer is cheapest and capable of answering. The README is explicit that the internal ladder, resolving a route, executing a captured endpoint, calling a standard adapter, opening a browser, reusing local cookies, inspecting HAR, capturing a new route and indexing it, is hidden behind that single hole.

That is a real architectural decision worth naming. Hiding the ladder means an agent cannot accidentally depend on the fact that today the answer came from a cached endpoint rather than a browser. It also means you cannot force a layer from the agent side; the runtime picks. If you want to see the ranking, you use the debug commands instead.

The second half is the receipt. Every operation produces a pointer-only, wallet-signed receipt that points at values (a URL, a value:ptr, a sha256: address) and carries a signature from your key, never the secret value itself. Credential fills dereference locally. The README claims the credential-redaction invariant held across all 24 sites in its live adversarial corpus, including the blocked ones. Receipts are Ed25519-signed today, and the README says stronger authorization and provenance schemes are an active research direction with specifics deferred to a forthcoming whitepaper. Treat the signing scheme as current-state, not settled.

The contract surface is machine-readable, which is the useful part for anyone auditing the boundary:

Installing the CLI and getting one real result

The repository is a Bun monorepo (bun.lock, bunfig.toml, workspaces listed in package.json), and the client boundary is MIT. The README does not spell out a single global install command, so the safest reading is that you work from the repository with Bun, or consume the published SDK package. What the README does give is the front door and the debug pair, which is enough to test the claim on your own domains.

The primary path is one flat command. You pass a task in natural language, optionally pinned to a URL:

unbrowse "the top Hacker News stories with points"
unbrowse "the top Hacker News stories with points" --url "https://news.ycombinator.com"

To inspect the ranking rather than the answer, the README lists a debug pair: resolve returns ranked routes for an intent and URL, and execute runs the chosen one.

unbrowse resolve --intent "..." --url "..."
unbrowse execute

Health is a single command, and it is the first thing to run after any setup change:

unbrowse health

If you prefer the SDK, the README gives this TypeScript example, which fills the same hole the CLI does:

import { createHole } from "unbrowse/sdk";

const hole = createHole();
const result = await hole.fill({
  intent: "get the top Hacker News stories with points",
  url: "https://news.ycombinator.com",
});

To read the live bridge contract itself, the README points at one endpoint:

curl https://beta-api.unbrowse.ai/v1/contract/surface

The local backend pieces are described by docker-compose.yml, which names the project unbrowse-local and runs pgvector/pgvector:pg16 as the postgres service with a postgres-migrate profile that executes bun install --frozen-lockfile and bun run db:migrate. The .env.example lists the keys a first run expects, including EMERGENTDB_API_KEY, OPENAI_API_KEY and an auto-generated UNBROWSE_API_KEY.

Where the API-native path fails

The README does not hide this, which is to its credit. On a live-harvested adversarial corpus of 24 sites, mined each run from r/webscraping plus curated vendor-gated, SPA and GraphQL targets, the shipped binary's API-native resolve to execute path covers 12/24 (50%) on retrieval. The misses are concentrated on JavaScript-challenge and commercial anti-bot gates such as Cloudflare and DataDome, and those route through the browser-capture path rather than the thin API path.

Read that carefully before you plan around it. Half the adversarial corpus does not resolve through the fast path. If your targets are ordinary product pages, dashboards or public APIs, the fast path is the one you will hit. If your targets are the sites practitioners complain about on r/webscraping, you are buying the browser fallback, and the browser fallback is the thing Unbrowse exists to avoid. The README records anti-bot misses with their vendor class and never relabels a miss as a pass, and the benchmark gates exit 0 only when the run was honest. That methodology is more trustworthy than the headline number.

There is a second boundary that is not a bug but a design choice. The README states plainly that the backend owns the route graph, ranking, settlement and recursive contract compilation, while the client sees only typed holes, approvals, pointer-only receipts and wallet-sealed values. You can audit what runs on your machine. You cannot audit how routes are ranked or how settlement is computed. docs/OPEN-SOURCE-NOTICE.md is where the project says the exact open/private split lives, and that file is the first thing to read if the split matters to your review.

The speedup claim and what it actually measures

The README's headline result is a 3.6x mean speedup and 5.4x median speedup across 94 live domains, measured when warmed cached routes replace browser automation, with sharply lower token use because the agent receives structured data instead of a page dump. The source is arXiv:2604.00694, and the project points to docs/benchmarks.md for current release-coverage methodology, corpus shape and rubric.

The gap between mean and median is the interesting part. A median of 5.4x against a mean of 3.6x means the distribution has a long tail of slow cases pulling the average down, which is consistent with the adversarial coverage number: some domains fall back to the browser and cost roughly what they always cost. The claim is warm-cache, so the first call on any domain pays the capture cost. Any evaluation you run should separate cold from warm, or you will reproduce the mean and wonder why it is not the median.

Unbrowse compared with a scraping API

The closest mental model is a hosted scraping API such as Firecrawl, and the repository acknowledges that lineage directly: one of the workspace packages is packages/firecrawl-shim. The difference in approach is what gets reused. A scraping API takes a URL and returns content, and every call is independent. Unbrowse takes an intent and a URL, learns the first-party route behind that site's UI, and reuses the learned route on later calls, so the second and subsequent calls skip the browser entirely.

That shift moves the hard problem from rendering to route maintenance. A scraping API absorbs anti-bot changes on its side and charges per call. Unbrowse keeps capture and execution local by default and makes routes maintained assets, with the README describing indexers being paid when their routes are reused and site-owner splits supported where claimed. That is a compensation surface, not just a cache, and it means route freshness becomes your operational concern in a way it is not with a hosted API. The shim packages also suggest Unbrowse can stand in for those tools rather than compete with them head-on, which is the more realistic deployment.

Maintenance, licence and the parts you cannot see

The repository is not archived, and the last push was on 2026-09-14, the same day as the v11.4.4 release. Three releases landed that day (v11.4.4, v11.4.3, v11.4.2), which tells you the release cadence is fast and the surface is still moving. The monorepo package.json carries version 11.4.0 while the releases are at 11.4.4, so expect the package version and the release tag to drift.

The client runtime, CLI bridge, SDK, drop-in adapters and wallet/auth/signing layer are MIT and readable. The backend is not, and the README frames that as company route IP behind typed contracts. For a team that needs to audit everything that touches credentials, the MIT client is the relevant artifact, and the pointer-only receipt design is what makes that audit tractable. For a team that needs to audit ranking or settlement, the documentation does not provide it. This is a description of the split, not legal advice; read LICENSE and docs/OPEN-SOURCE-NOTICE.md against your own requirements.

Upgrade cost is the open question the README does not answer. Three releases in one day on a fast-moving contract surface means the hole contract and the receipt shape are the things to watch across versions, and the README does not document a rollback procedure or a compatibility guarantee for either.

Editorial conclusion

Adopt Unbrowse if your agents repeatedly hit the same sites and you want typed, cached routes instead of a fresh browser session per call, and you accept that the ranking and settlement backend stays closed while the client does not. Do not adopt it if your targets sit behind Cloudflare or DataDome style JavaScript challenges, because the README reports the shipped API-native path covering 12/24 on a live adversarial corpus and the misses concentrate exactly there. Before committing, run `unbrowse resolve --intent "..." --url "..."` and then `unbrowse execute` against your own top three domains and read the receipts, since those two commands are the whole claim.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. unbrowse-ai/unbrowse on GitHub