OpenSERP: a self-hosted SERP API for six engines, browser-rendered

The gap OpenSERP fills between paid SERP APIs and hand-written scrapers

Commercial SERP APIs charge per search and cover a fixed set of engines. OpenSERP takes the other route: you run the server, the searches cost nothing per call, and the engine list includes Yandex, Baidu and Ecosia alongside Google, Bing and DuckDuckGo. The README states the pitch plainly: no API keys, no per-search billing, live structured results on localhost. The project is written in Go and published under MIT.

The audience is narrow but real. Someone building retrieval for an LLM or an agent needs a search call that returns text, not an HTML page. Someone tracking SEO rankings needs the same query run repeatedly against the same engine and a stable shape to diff. Both groups currently choose between paying a vendor or maintaining a private scraper. OpenSERP is the second option with the maintenance partly done for you: the selectors, the browser rendering and the JSON schema are already written, and the repository is the thing you fork when they break.

The README also points at a hosted version at openserp.org/cloud with the same API. That matters for evaluation, because it means you can compare response shapes before deciding whether to run the server yourself.

How a request flows from CLI or HTTP into a merged result set

The server exposes dedicated endpoints per engine plus a combined one. The README's first request example is the combined path: GET /mega/search with engines=bing,google, text=golang+vs+rust, extract=1 and mode=any. The mode parameter controls how the fan-out is resolved. With mode=any, the response returns the first engine that responds, and the meta block records which engines answered and which failed. That is a deliberate trade: lower latency, fewer results, and a record of what was skipped.

Each engine produces results that are then clustered. In the sample response, a cluster object carries a canonical_url, a domain, the occurrences list with engine, rank and result_id, plus engines_count, best_rank and a score. So the merge step is not just concatenation. The same URL found by two engines collapses into one cluster with both ranks preserved, which is what makes cross-engine rank comparison possible at all.

Result objects carry more than title and link. The example shows id, rank, type, url, display_url, snippet, domain, favicon, position.absolute, engine, a domain_info block with tld and sld, and an optional classification block with content_type and source_hint. When extract=1 is set, an extracted object appears with title, format, content, mode_used and fetched_at. The content field in the sample is markdown, and the README lists JSON, Markdown, Text and NdJSON as output formats. The extraction step is what turns a result list into something you can feed to a model without a second fetch.

The README also lists SERP features as a field: AI summaries, answer boxes, people-also-ask and related searches. In the sample response serp_features is an empty array, so the shape is documented but the sample does not demonstrate populated content.

Getting a server running with Docker, Go install or a source build

Three installation paths appear in the README. The Docker route uses the published image and binds the port to loopback only:

docker run --rm -p 127.0.0.1:7000:7000 karust/openserp:latest serve -a 0.0.0.0 -p 7000

Note the split: the host side is bound to 127.0.0.1 while the container side listens on 0.0.0.0, which is the correct pattern for a service you do not want exposed. The alternative is docker compose up. The Go route is go install github.com/karust/openserp@latest followed by openserp search duckduckgo "open source serp api" --format markdown. From source, the README gives git clone, cd openserp, go build -o openserp . and ./openserp serve.

The serve subcommand takes -a for the bind address and -p for the port. The search subcommand takes an engine name, a quoted query, and --format. The README lists proxies, cache and a resilient mode among the features but does not show the flags or config keys for them, so treat those as documented-but-unspecified until you read the source or the pkg.go.dev reference linked in the badge.

Client integration is covered by official packages: @openserp/sdk on npm, openserp on PyPI, an MCP server at @openserp/mcp runnable with npx, and an n8n community node. The README says each client works against your self-hosted server by setting baseUrl, or against the hosted API by setting apiKey. That single switch is the cleanest part of the design: the same client code points at either backend.

Where OpenSERP breaks: markup drift, browser cost and mode=any semantics

The honest limitation is structural. OpenSERP renders pages in a browser and parses what comes back. Search engines change their result markup without warning, and when that happens the parser for that engine returns nothing or returns the wrong fields. The meta block in the sample response exists precisely because this is expected: engines_failed is a first-class field. The README does not describe a selector-update process, a canary, or a compatibility guarantee with any engine's current layout. If you depend on one engine, you own that risk.

mode=any makes the risk sharper. Returning the first engine that responds means a request can succeed with results from a single engine while the others failed silently unless you inspect meta.engines_failed. Code that reads only results will not notice. If your application needs coverage from a specific engine, mode=any is the wrong setting.

Browser rendering also costs more than an HTTP client. The sample response reports took_ms of 720 for a two-engine request with extraction, and that is one measured example from the README, not a benchmark. Extraction adds a fetch of each target page on top of the SERP fetch, so extract=1 multiplies the work per query. The README lists a cache and a resilient mode but gives no numbers for either, so capacity planning has to be done against your own workload.

Finally, the repository sits under karust/openserp while the SDKs, MCP server and n8n node live under the openserpapi organization. Two release tracks have to be kept compatible, and the README does not state a versioning policy between them.

OpenSERP against SerpApi and against writing your own scraper

The obvious comparison is a commercial SERP API such as SerpApi. The difference is not features, it is where the failure lands. A paid API absorbs markup changes, handles proxy rotation and returns a schema it commits to; you pay per search and you get a support relationship. OpenSERP moves all of that to your machine. You get the same class of output, no per-call cost, and engine coverage the README claims paid APIs do not offer, but when Google changes its markup you are the one who patches the parser and redeploys. For a team running a few thousand queries a day, the arithmetic can favour self-hosting. For a product with an uptime commitment, the arithmetic usually does not.

The second alternative is a hand-written scraper. Here OpenSERP's advantage is concrete: six engines behind one schema, a clustering step that merges duplicate URLs across engines with per-engine ranks preserved, and an extraction path that returns markdown. Writing that yourself means writing six parsers, a merge layer and a readability pipeline before you get to your actual problem. The cost you accept in exchange is that the parsers are someone else's code with someone else's release cadence. The recent releases listed for the repository (v0.8.12, v0.8.6, v0.8.3) show active iteration, which cuts both ways: fixes arrive, and so do changes you have to track.

Licence, upgrade surface and what maintenance actually looks like

The repository is MIT licensed. That permits commercial use, modification and redistribution provided the copyright notice and licence text are retained. It does not grant rights to the search engines' content, and it says nothing about whether scraping a given engine is permitted under that engine's terms of service. That question is separate from the licence and is yours to answer. Nothing here is legal advice.

The practical maintenance cost is upgrade tracking. A Go module pinned to a tag, or a Docker image pinned to a digest rather than latest, gives you a controlled upgrade point. Between v0.8.3 and v0.8.12 there are three listed releases over roughly six weeks, so the cadence is frequent enough that pinning matters. If you use the official SDKs, you are tracking two version numbers: the server and the client package. The MCP server and n8n node add two more if you use them.

The other recurring cost is verification, not patching. Because engines change independently, a passing test suite on your side does not prove that a given engine still parses correctly today. A scheduled request per engine, checking that results is non-empty and that meta.engines_failed is empty, is the only way to know. The README does not ship such a check, so it is work you add.

Who should run OpenSERP and what to confirm before you do

Run it if you need search results on hardware you control, if the engines you care about include ones the paid APIs skip, and if you have someone who can respond when a parser breaks. The Docker one-liner and the Go install path both get you to a working endpoint quickly, and the shared JSON schema across six engines is the part that saves the most work.

Do not run it as the only search backend for a latency-sensitive product, and do not treat mode=any as a reliability feature. It is a latency feature with a coverage cost, and the failure is visible only in meta.engines_failed. If you need guaranteed coverage from a named engine, request that engine directly and handle the empty case.

Before committing, do three things. Start the image, run one request against each engine you intend to use, and confirm the response shape matches what the README documents, including the clusters and extracted blocks. Check that your deployment pins a version rather than latest. And read the MIT licence text alongside your own redistribution plans, because the licence covers the code and not the search results it returns.

Editorial conclusion

Adopt OpenSERP if you need search results on infrastructure you control, especially for engines that paid SERP APIs do not cover, and if you can absorb the cost of re-checking selectors when a target engine changes its markup. Do not adopt it as the only search backend for a product with a hard latency budget or a contractual requirement for a supported upstream. Before committing, run the Docker image, send one request per engine you care about, confirm that the JSON schema holds, and read the MIT licence terms against your own redistribution plans.

Official sources

1. karust/openserp on GitHub
2. License: MIT
3. Project website
4. README
5. Releases