Google Search API: a guide to pulling SERP data as JSON instead of scraping Google

What the repository actually is

This repository is a single README and nothing else, and it says up front what it is: a production-ready Google Search API scraper built on top of a scalable Google web scraping API, and a demonstration of extracting Google search results as structured data through a managed API that takes on proxy rotation, JavaScript rendering, anti-bot protection and CAPTCHA solving for you.

So this is documentation, not an implementation. There is no library to install, no code to compile, and no client in the repository. What there is, is an integration guide for the ScrapingBee service, with the endpoint, the parameters, the examples in three languages, the error codes and the response shape.

That is worth reading on its own terms, because the request contract it documents is a reasonable design for a scraping API, and the guide is explicit about what the service takes on your behalf. The five things the API handles are IP rotation, anti-bot bypass, CAPTCHA solving, JavaScript rendering and result normalisation. The five things you send are the target query, location parameters, language parameters, device type and optional rendering options.

The primary language is recorded as unknown, which is accurate for a repository containing a single Markdown file, the licence is not a standard identifier, the default branch is master, and the homepage is the ScrapingBee Google feature page. The last push was on 2026-04-08 with the project not archived, and the repository publishes no GitHub releases.

The topic list is long and mostly variations on the same phrase, which tells you the repository is written to be found by people searching for a Google search API rather than for a tool. It includes google-api, google-scraper, google-scraping, google-search, proxy-scraper, serp-api, serp-scraper, web-scraping-python, how-to-scrape-google and public-api.

There is also a related projects list of things a similar thing might be called, including SERP APIs and proxy scrapers, which is a hint that the author is competing for search terms rather than shipping code.

The request flow, and why the arrows matter

The request flow is drawn as a single line in the README:

Client to Google Search API Scraper, to Proxy Rotation, to Google, to Structured JSON Response.

And the architecture overview at the end of the guide expands it into six stages: Client Application, then Google Search Scraper API, then Proxy Rotation, then Google SERP Retrieval, then Parsing Engine, then Structured JSON Response.

The two versions are not identical, and the difference is informative. The short version starts at proxy rotation, which frames the service as a proxy you point at Google. The long version inserts the SERP retrieval and parsing stages, which is a more accurate picture of what actually happens: the service fetches the result page from Google, parses it, and hands you structured fields.

That parsing stage is the part with the real value, and it is the part the surrounding prose is selling. The argument the README makes is that scraping Google manually is complex and fragile, and that the traditional approaches require proxy management, IP rotation, CAPTCHA handling, browser automation and constant selector maintenance. Google actively detects automated traffic, and the infrastructure quickly becomes expensive and unstable.

The selector maintenance point is the one developers recognise immediately. A scraper that reads the results page has to track Google's HTML, and when the markup changes, the scraper breaks, and somebody has to fix it before your pipeline works again. A managed service takes that on.

The abstraction the guide describes is that you make a request and get back clean JSON, and the whole point of the request flow diagram is that the messy parts sit in the middle, between your query and your JSON.

What the diagram does not show is the cost and the failure modes of that middle, which is where the rest of the guide is more useful.

The endpoint and the required parameters

The API endpoint is:

GET https://app.scrapingbee.com/api/v1/

And for Google search scraping specifically, the query string form is:

https://app.scrapingbee.com/api/v1/?api_key=YOUR_API_KEY&search=google&q=QUERY

So the API is a single GET endpoint with the search engine selected by a parameter, and the first example adds a search engine of google and a q parameter with the query.

The required parameters are three, and they are small. The api_key is the authentication key. The search=google parameter says the request should go to the Google search scraper rather than another engine. And q holds the search query.

That the engine is a parameter rather than a separate endpoint is the design decision worth noting. One endpoint, one set of parameters, and the service dispatches internally. It also means the parameter is what distinguishes a Google request from a request for a different engine, so a client written against the guide has to remember to set it.

The optional parameters extend the request in four ways, and each one maps to a way that a raw Google request is ambiguous.

country_code is geolocation targeting, with examples given for us, uk and de. This is not a cosmetic option, because the same query returns different results in different locations, and if you are collecting SERP data for analysis, the location of the result is part of the datum. Leaving it out means you get whatever the proxy's location returns, which is not a fixed thing.

language is the language of the search results, and it is separate from country_code. That separation is correct, since a user in Switzerland might search in German and want Swiss results, or French results, and tying the two together would remove that choice.

device specifies desktop or mobile. This matters because Google serves different result pages to desktop and mobile clients, with different layouts and different features, and a scraper that ignores it is scraping one of them and calling it the result.

render_js enables JavaScript rendering when required, and premium_proxy uses a different proxy pool for higher reliability. The last of those is a quality-versus-cost knob, since a proxy pool built for reliability is not the default pool.

The examples, in three languages, and the response shape

The guide gives the same request three times, which is the right way to write it, because the interesting variation is not the query but the client.

In curl it is a single GET with the parameters in the query string:

curl "https://app.scrapingbee.com/api/v1/?api_key=YOUR_API_KEY&search=google&q=web+scraping&country_code=us&language=en"

In Node it uses the official ScrapingBee client, with the URL set to the Google search page and the parameters passed as an object:

const { ScrapingBeeClient } = require('scrapingbee');

const client = new ScrapingBeeClient('YOUR_API_KEY');

async function googleSearch() {
    const response = await client.get({
        url: 'https://www.google.com/search',
        params: {
            search: 'google',
            q: 'web scraping tools',
            country_code: 'us',
            language: 'en'
        }
    });

    console.log(response.data);
}

googleSearch();

And in Python it is a plain requests.get with a params dictionary:

import requests

params = {
    "api_key": "YOUR_API_KEY",
    "search": "google",
    "q": "google scraping api",
    "country_code": "us",
    "language": "en"
}

response = requests.get("https://app.scrapingbee.com/api/v1/", params=params)

print(response.json())

import requests

params = {
    "api_key": "YOUR_API_KEY",
    "search": "google",
    "q": "google scraping api",
    "country_code": "us",
    "language": "en"
}

response = requests.get("https://app.scrapingbee.com/api/v1/", params=params)

print(response.json())

The three are worth comparing rather than reading separately. The curl form puts everything in the query string. The Node form sets the target URL explicitly and puts the service options in a params object, which is the shape the client library expects. The Python form omits the target URL entirely and passes the endpoint with the parameters as a dictionary, letting requests do the encoding.

So the guide is showing that the service supports both styles, which means the choice of client library is not forced by the API design.

The response is JSON, and the example is given in full:

{
  "organic_results": [
    {
      "position": 1,
      "title": "Best AI Tools in 2024",
      "link": "https://example.com/best-ai-tools",
      "snippet": "Discover the top AI tools for productivity and automation."
    },
    {
      "position": 2,
      "title": "AI Software Comparison",
      "link": "https://example.com/ai-software",
      "snippet": "Compare leading AI software solutions."
    }
  ],
  "search_information": {
    "query": "best ai tools",
    "country": "us",
    "language": "en"
  }
}

The structure is straightforward. There is an organic_results array, and each entry has a position, a title, a link and a snippet. And there is a search_information object carrying the query, the country and the language back to you.

That second object is the part that matters most for data collection, and it is the part most scrapers forget. When you collect SERP data at volume, you are collecting it over weeks, and the country and language the result was produced for are attributes of the observation. Without them, a dataset collected from several countries is a dataset where you cannot tell which country a row came from, and the first thing that goes wrong is an aggregate that mixes them.

The list of extractable data is broader than the example shows: organic results, paid ads, featured snippets, knowledge panels, People Also Ask, related searches, local pack results and shopping results, all in structured JSON format.

Errors, quotas and the practices that keep a pipeline alive

The error section lists four typical responses, and the codes are the standard HTTP set with a specific meaning in each case.

A 401 is an invalid API key. A 403 is access denied. A 429 is rate limit exceeded, and a 500 is a server error.

Then there is one line of advice, and it is the most practically useful sentence in the guide: for production systems, implement retry logic and monitor usage quotas.

That line implies several things that the rest of the guide does not spell out. A 429 is a rate limit, which means the service is metering you, which means a crawler that fires a request per page without thinking will hit it. And a 500 is a server error on the service's side, which is exactly the case where retrying is correct, which is why retry logic is called out separately from the key and permission errors, where retrying would just waste requests.

The best practices section expands the same idea into five items: use country targeting for accurate SERP data, implement retry logic, cache repeated queries, monitor rate limits, and use premium proxies for high-volume workloads.

Two of those deserve a note. Country targeting is listed first, which is a hint that the default geolocation is not what a data collector usually wants, and the fact that a proxy-based service will follow the proxy's location means an unspecified country_code produces results that vary with your infrastructure rather than with your query. And caching repeated queries is the difference between a SERP collection job that costs a fixed amount and one that costs a variable amount, since search results for a given query in a given country change slowly but not never.

The premium proxy line is the other one. It appears both in the parameter list and in the best practices, and it is tied to high-volume workloads, which suggests it is a paid tier rather than a free upgrade.

How to evaluate it, and what to check first

There are four things to consider, and the first is that this is a commercial product documented in a repository rather than a piece of software you can read.

Everything here points at a managed service. The endpoint is a ScrapingBee endpoint, the Node example uses the ScrapingBee client library, and the guide is the service's own documentation. So you are evaluating a paid service with a credential, and the repository is the manual. That has two consequences: the code examples are the fastest way to see the shape of the contract, and you should look at the service's own pricing and terms rather than expecting anything in this repository to be free or self-hosted.

Second, the parameters that change your data. The country_code and language combination determines which results you get, and the device parameter determines whether you are looking at the desktop or the mobile result page. If you are building a rank tracker or an SEO monitor, those three are not optional, and getting them wrong produces a dataset that looks fine and is wrong.

Third, the cost of the abstraction. The service does the proxy rotation, the CAPTCHA solving and the JavaScript rendering, and it charges for that. For a small, occasional query that is obviously worth it. For a high-volume collection job, the premium proxy tier and the rate limits become the design constraints, and you should decide whether you are building a scraper or a service client, because the two have different cost structures.

Fourth, the parsing is a black box. Whatever the service returns for a field is whatever its parser extracted, and if Google changes the page the service absorbs that and you never see it. That is the benefit and the risk: your pipeline does not break, but you also cannot tell that the data changed shape, so validate the output periodically rather than trusting it indefinitely.

What you get is a clean, well-documented request contract with sensible defaults, examples in the three languages most people use for scraping, and honest advice about retries and rate limits. For a team that needs SERP data now and does not want to run a browser farm, that is a reasonable deal, provided you read the pricing and set the location parameters deliberately.

Editorial conclusion

Adopt this guide, and the service it documents, if you need Google search results as structured data now and would rather not run proxies and a browser yourself. Do not treat it as an open source library, because the repository contains a single README and everything depends on a commercial API with a key and a quota. Before you commit, set country_code, language and device deliberately on every request, since an unspecified location makes your results depend on the proxy rather than on your query, and budget for the rate limit and the premium proxy tier if you are collecting at volume, because the two together are what actually determine the cost of a crawl.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. ScrapingBee/google-search-api on GitHub