pi-research: local web search and a knowledge store for coding agents

What pi-research actually solves

Most AI search providers cap free searches and charge after a quota, and they serve results from their own index on their own servers. pi-research removes the provider and the quota by searching and scraping locally through DuckDuckGo in a stealth browser. The only cost is LLM tokens. This matters for developers who run coding agents like pi, Claude Code, or Codex and want research capability without a monthly search bill. The tool is also a knowledge store: every finding can be saved to a local LanceDB database that seeds future runs, so repeat questions get faster and cheaper answers. The target user is someone building or running an agent that needs to read web content, not someone who wants a hosted search API.

How the research loop is structured

The README describes a research run as a loop through agent teams. A coordinator plans and starts with a search. Researcher agents scrape and read in parallel. A research lead decides whether to go another round, then writes a report from every report collected. The output is one cited Markdown report, with findings optionally saved to the knowledge store. There are three depth levels: normal, deep, and ultra. These set the team size and the number of rounds. A natural-language request is enough; the tool picks the right depth. This is a multi-agent architecture, not a single search-and-summarize call. The parallel scraping and the lead's round decision are what make it a research tool rather than a simple search wrapper.

Installation: the npm 12 trap

Installation is straightforward on paper. As a pi extension: `pi install npm:@lincoln504/pi-research`. Standalone, as an agent skill: `npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3` followed by `pi-research skill install`. The catch is npm 12. On npm 12 or newer, npm skips dependency install scripts by default. The flag `--allow-scripts=better-sqlite3` is required to build the SQLite module for search. Without it, every search fails with a missing-module error. The README warns that `pi install` passes no flags to npm, so on npm 12 you must bracket any install or update with a config setting: set `allow-scripts=better-sqlite3`, run the install, then delete the config. Leaving the setting in place can break installs of other projects with git dependencies, per npm/cli#9783. This is a genuine operational hazard, not a footnote.

The knowledge store and read-only design

The knowledge store is a local LanceDB database. Findings from research runs can be saved there, scoped globally or per project. Future runs can seed from the store, which is how repeat questions get faster and cheaper answers. The store is searchable, so it acts as a persistent memory for agent research. The read-only design is a deliberate safety choice. The research agent cannot run shell commands or write, edit, or delete anything. The rationale is that the agent's job is reading untrusted web pages, and prompt injection picked up mid-run has nothing to act on. This is a strong stance for a tool that scrapes arbitrary pages. The trade-off is that you cannot use pi-research to perform any action on the web beyond reading, which limits its use for tasks like form submission or authenticated scraping.

Hard requirements and known failure modes

The requirements are specific. Node.js >= 22.19.0. An LLM with a 100k+ context window, either an API key or a local model. Internet access on a residential IP, because search, scraping, and YouTube transcripts get bot-blocked from datacenter, VPS, and cloud IPs. This is a hard constraint for anyone running agents on typical cloud infrastructure. The README also states that Cloudflare Turnstile and similar systems block scraping on some sites. A run compensates with a wide pool of search results to scrape, but that is a mitigation, not a guarantee. If your target sites are behind aggressive bot protection, pi-research will fail on those sites. The first install downloads the stealth browser engine, which takes a few minutes, so initial setup is not instant.

Alternatives and how they differ

The obvious alternative is a hosted search API like Tavily or Brave Search API. Those services run on their own infrastructure, have a monthly cap or fee, and return results from their index. pi-research differs by running search and scraping locally through DuckDuckGo, so there is no provider to pay and no server-side index. Another alternative is to use a general-purpose scraping library like Puppeteer or Playwright directly, combined with an LLM call. That approach gives you full control over the scraping logic, but you must build the research loop, the multi-agent coordination, and the knowledge store yourself. pi-research packages all of that into one tool. The difference is that a direct scraping library is a building block, while pi-research is a complete research loop with a built-in store.

Maintenance, license, and package identity

The project is MIT licensed, with bundled third-party licenses listed in docs/THIRD-PARTY-NOTICES.md. The npm package `@lincoln504/pi-research` is the stable channel and is kept current with breaking pi changes. A git install is the development channel, with the latest commits and breaks first. The README explicitly warns that the unscoped `pi-research` package is unrelated and deprecated, so you must use the scoped name. Maintenance cost is low on the license side, but the dependency on a stealth browser and DuckDuckGo means you inherit changes in those systems. If DuckDuckGo changes its bot detection or the stealth browser engine breaks, the tool will need updates. The project has no recent releases listed, so you should check the repository's activity before relying on it for production.

Editorial conclusion

Adopt pi-research if you run a coding agent on a residential IP, want to avoid per-search fees, and need a read-only research loop that saves findings to a local store. Skip it if your agent runs on a datacenter IP, if you cannot tolerate npm 12 install quirks, or if you need to scrape sites behind Cloudflare Turnstile. Before adopting, verify your Node.js version (22.19.0+), confirm your LLM has a 100k+ context window, and test a search on a site you actually need. The project is MIT licensed, but the npm 12 allow-scripts dance and the residential IP requirement are real constraints that will shape your deployment.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository