Oxylabs AI Studio Python SDK: Prompt-Driven Scraping With a Credit Ceiling

The Problem: Extraction Logic Written Per Site

Most scraping code encodes assumptions about one site's DOM. A selector for a price, a selector for a title, a loop that follows pagination. When the site ships a redesign, the code breaks and someone rewrites it. The AI Studio SDK moves that logic to a prompt. In the crawler example in the README, the instruction is the string "Find all pages with proxy products pricing" paired with a starting URL and a return_sources_limit of 3. The SDK does not contain a selector for anything. The service decides which pages matter and what to pull from them.

That reframes the job. You are no longer maintaining parsers. You are writing instructions and, where you need typed output, JSON schemas. The audience is engineers building data pipelines over sites they do not control, and teams wiring fresh web content into an LLM context window. The repository topics list ai-grounding and ai-training alongside the more conventional web-scraping tags, which matches the framing: the output is meant to be consumed by a model, not only by a database.

Four Entry Points and What Each One Actually Does

The package exposes separate apps rather than one client with a mode flag. AiCrawler.crawl takes a starting URL and a user_prompt and returns items up to return_sources_limit, which defaults to 25. AiScraper.scrape takes a single URL and returns that page in the requested format. BrowserAgent.run takes a URL plus a user_prompt that can describe interaction, and the README's example prompt is explicit about it: "Find if there is game 'super mario odyssey' in the store. If there is, find the price. Use search bar to find the game." That is a sequence of actions, not a parse. AiSearch.search runs queries and optionally returns markdown content with each result. AiMap.map takes a payload dictionary with url, search_keywords, user_prompt, max_crawl_depth and limit, which suggests it is for enumerating a site's structure before deciding what to crawl.

The distinction matters when you are budgeting. A crawl with return_sources_limit of 25 is not the same request shape as a scrape of one URL or a browser agent that types into a search box. Choosing the wrong entry point is the most likely way to overspend.

Schemas, Output Formats, and generate_schema

Output format is a required decision, not a default you can ignore. AiScraper.scrape accepts json, markdown, csv, screenshot or toon, with markdown as the default. AiCrawler.crawl accepts json, markdown, csv or toon. BrowserAgent.run adds html and screenshot to that set. The documentation is blunt about the coupling: schema is required if output_format is json, csv or toon. Ask for json without a schema and the call is malformed.

To avoid hand-writing schemas, AiScraper and BrowserAgent both expose generate_schema. The scraper example passes a prose description ("want to parse developer, platform, type, price game title, genre (array) and description") and prints the returned schema, which is then fed into the scrape call. That is a two-call pattern, and the second call is the one that consumes credits against a real page. The genre field being described as an array shows the generator handles nested types, but the README does not document how to correct a schema it gets wrong, so plan on inspecting the generated object before you send it.

The screenshot format is worth noting separately. It returns an image rather than parsed data, which makes it useful for visual verification and useless for a data pipeline without a separate OCR or vision step.

Getting It Running: Install, Key, First Call

Requirements are Python 3.10 or above and an API key. Installation is a single command:

pip install oxylabs-ai-studio

Every app is constructed with the key passed directly: AiCrawler(api_key="<API_KEY>"), AiScraper(api_key="<API_KEY>"), BrowserAgent(api_key="<API_KEY>"), AiSearch(api_key="<API_KEY>"), AiMap(api_key="<API_KEY>"). The README shows the literal string placeholder rather than an environment variable, and it does not document an env-var fallback. If you keep keys in the environment, you are passing os.environ values into the constructor yourself.

A minimal working call, following the README's crawler example, sets url, user_prompt, output_format, render_javascript, return_sources_limit and geo_location, then iterates result.data and prints each item. Two parameters deserve attention before your first production run. render_javascript defaults to False in both crawl and scrape, and AiScraper accepts the string "auto" so the service detects whether rendering is needed. Leaving it False on a client-rendered site is a plausible cause of empty results. max_credits is documented as an optional ceiling on credits for the crawl call. That is the only spend guard mentioned in the material, and it is listed for the crawler, not for the other apps.

Where the Design Gets Thin

The README documents parameters and examples. It does not document error types. There is no section on what a failed scrape returns, whether rate limits produce a retryable exception or a plain error, or how partial crawls report pages they could not reach. For a library whose whole job is fetching unreliable remote pages, that is the largest gap in the supplied material, and you should treat exception handling as something to discover by reading the source rather than the docs.

The credit model is also underdescribed. max_credits appears on the crawler as an optional integer, with no statement of what a credit is worth, how it is computed per page, or what happens when the ceiling is hit mid-crawl. If you are running this at volume, that is the number your finance conversation depends on, and it is not in the README.

There is a second boundary that is not a documentation gap but a property of the architecture. Every call sends the target URL and your prompt to Oxylabs. Pages behind a login, internal dashboards, or anything under a data processing agreement that forbids third-party transfer are out of scope, regardless of how well the SDK works. The SDK is a client for a hosted service, and no amount of local code changes that.

Alternatives and the Real Difference in Approach

The repository topics name firecrawl-alternative and tavily-alternative, so the author expects those comparisons. The distinction is not feature lists. It is where the extraction logic lives and how the browser is controlled.

Against a local stack of requests plus BeautifulSoup or Playwright, the difference is total. A local stack runs on your machine, costs nothing per page, and gives you full control over retries, caching and headers. It also requires you to write and maintain a parser per site, and to run a headless browser yourself when JavaScript rendering is needed. The SDK trades that maintenance for a per-request bill and a dependency on someone else's uptime. If your targets are static HTML with stable structure, the local stack wins on cost and on data residency, and there is no reason to involve a prompt.

Against Firecrawl or Tavily, the shape is closer: hosted extraction with a natural language interface. The differences visible in this material are the breadth of entry points and the proxy layer. The SDK exposes four distinct operations (crawl, single-page scrape, agentic browsing, search) plus a map operation, and geo_location is a first-class parameter on multiple methods, pointing at Oxylabs' proxy infrastructure and the documented list of supported proxy location values. Tavily is oriented toward search results for LLM grounding; Firecrawl toward crawl and scrape with markdown output. Neither positioning is a quality claim, and the README offers no benchmark against either, so the honest comparison is about which operation set matches your workflow and which vendor's regional coverage you need.

Maintenance, Versioning, and the MIT Licence

The package is MIT licensed, which permits commercial use and modification with the licence text retained. That covers the SDK code in this repository. It does not cover the AI Studio API, which is a paid hosted service governed by separate terms, and it does not grant any rights over the pages you scrape. Those remain the site's terms and your local law. Nothing here is legal advice; if you are scraping at commercial scale, that question is separate from the licence file.

The version history shows v0.2.19 released in November 2025, with the repository still receiving pushes as of August 2026. A 0.x version line means the maintainers have not declared the API stable, and the parameter surface is broad enough that a breaking change would touch every call site. Pin the version in your requirements file rather than tracking latest, and read the release notes before upgrading. The practical upgrade cost is low if you wrap each app behind a thin function of your own, and high if AiCrawler, AiScraper and BrowserAgent calls are scattered through your codebase with their parameters inline.

Editorial conclusion

Adopt this SDK if you need prompt-driven extraction from pages that require JavaScript rendering, geo-targeted requests, or multi-step browser interaction, and you are willing to run it as a hosted API with a credit budget. Do not adopt it if your targets are static HTML that requests plus BeautifulSoup already parse, or if you cannot send page content to a third party. Before committing, verify what a credit costs for your page volume, confirm that the geo_location values you need appear in the Oxylabs proxy location list, and check whether your chosen output_format requires a schema, because json, csv and toon all do.

Official sources

1. License: MIT
2. oxylabs/oxylabs-ai-studio-py on GitHub
3. Project website
4. README
5. Releases