AgentQL: A Query Language for Web Automation, With a Hosted Service Behind It

The Problem AgentQL Targets: Selectors That Break

Every browser automation script is built on a bet that the page will look the same tomorrow. A Playwright script that clicks a button by CSS class breaks when a frontend refactor renames that class. An XPath that walks a table's DOM path breaks when a wrapper div appears. The maintenance burden is not in writing the script, it is in re-finding the elements after each deploy of the target site. AgentQL's answer is to stop addressing elements by position or attribute and address them by meaning instead. The README describes the query language as using natural language queries to pinpoint data and elements, and it lists resilience to UI changes as a feature, claiming queries work regardless of how a page's structure changes over time. That is the pitch: a selector written as a description of what you want, not where it lives. The audience is developers who already write Playwright automation and are tired of selector churn, and teams building agents that need to read arbitrary pages rather than one known page. The README also names RPA and scraping among the repository topics, which fits the same group.

How Query Resolution Actually Flows

The architecture visible in the material is a client-server split. On one side there is a query language, described in the docs as letting you define queries with natural language. On the other side there is a REST API endpoint for executing queries without an SDK, plus Python and JavaScript SDKs that wrap the same capability. The SDKs integrate with Playwright, so the browser is driven locally while the query is resolved elsewhere. The README states that queries can define structured output by the shape of the query, and that transforms and extraction happen inside the query itself, which means the response shape is determined at query-writing time rather than parsed afterwards in your script. There is also a debugger browser extension, distributed through the Chrome Web Store, that lets you iterate on a query against a live page and see what it matches. A playground is offered as well, and the README says it can export Python scripts. The practical consequence of this design is that the hard part, mapping a description to a DOM node, is not running in your process. The README does not describe an on-device model or a local resolver, so the reasonable reading is that the SDKs call out to AgentQL's service for that step. That is the single most important thing to confirm before you build on it, and the README does not settle it.

Installing the Python SDK and Writing a First Query

The README points to the Python SDK installation page in the docs and to a quick start rather than embedding install commands, so the exact pip invocation is not in the material I have. What is in the material is the shape of the workflow. You install the SDK, you write a query in the AgentQL query language, and you run it against a page through Playwright. The README's example table gives the concrete entry points: examples/python/first_steps is the getting started script, examples/js/first-steps is its JavaScript counterpart, and there are named examples for specific tasks such as close_cookie_dialog, close_popup, compare_product_prices, get_by_prompt and infinite_scroll. Each of those has a Python script, most have a Google Colab notebook, and several have a JavaScript version. Those directories are the fastest way to see real query syntax without reading the language reference, because the README does not reproduce a query inline. The REST API path exists for the case where you do not want an SDK in the loop at all, which matters if your runtime is not Python or JavaScript. The debugger extension is the iteration tool: you install it in Chrome, open the target site, and refine the query against the live DOM before pasting it into a script. The README does not state authentication requirements for the API, so treat the quick start page as the source for whatever key or account setup is needed.

Where AgentQL Is the Wrong Choice

The dependency on a hosted resolver is the limitation that decides most adoption questions. If your scraping runs inside a network that cannot reach external endpoints, or if the pages you process contain data you are not permitted to send to a third party, this design does not fit regardless of how well the queries work. There is no local mode described in the README. The second limitation is determinism. A CSS selector either matches or it does not, and you can reason about that in advance. A natural language query is resolved by a model, and the README's own framing of resilience to UI changes implies that the resolver is making a judgement call each time rather than following a fixed rule. That is the feature, but it also means the same query can resolve differently across page states, and a page with two plausible matches for a description gives the resolver a choice you did not make explicitly. The README does not discuss disambiguation behaviour or how to pin a match. Third, this is a poor fit for pages you control. If you are testing your own application, you know the markup, and Playwright's own locators with test ids are more precise and carry no external dependency. AgentQL earns its cost when the pages are not yours and change without notice. Finally, the repository shows no retrieved releases in the material I was given, so there is no version history here to judge how the query language has evolved or how breaking changes are communicated. Verify that on PyPI rather than assuming stability.

AgentQL Versus Scrapy, BeautifulSoup and Plain Playwright

The closest comparison is not another AI scraping tool, it is the traditional stack. Scrapy, BeautifulSoup and plain Playwright all share one property: the selector is a literal expression evaluated by the browser or the parser, and the result is fully determined by the page bytes. Nothing is inferred. You pay for that with fragility, because a structural change to the target page silently breaks the selector, often without an error, just with an empty result. AgentQL inverts the trade. You describe the target and let a service resolve it, which the README says keeps queries working as the UI changes. You pay with an external call per resolution, a dependency on someone else's uptime, and a result that is inferred rather than proven. For a one-off scrape of a stable page, the traditional stack is cheaper and easier to debug. For a pipeline that reads dozens of similar sites and must survive their redesigns, the inversion starts to pay for itself, which is exactly what the README means by cross-site compatibility: the same query across different sites with similar content. The other comparison worth naming is writing your own LLM prompt to extract HTML. That approach also infers, but it hands you raw model output to parse and validate, and it has no structured output contract. AgentQL's claim is that the query shape defines the output shape, which removes a parsing layer if it holds up.

Maintenance, Licensing and What to Verify

The repository is MIT licensed, which is permissive for the code in it, but the licence line covers this repository, not necessarily the hosted query service or the query language specification. Those may carry separate terms, and the README does not say. That distinction matters more than usual here because the SDK is thin: the value sits in the resolver, and an MIT client for a commercial endpoint is a common and legitimate pattern that still leaves you with a service dependency and whatever its pricing and rate limits are. On maintenance, the material shows a recent push to main and no retrieved releases, so I cannot describe a release cadence or a deprecation policy. The upgrade surface you actually care about is the query language and the SDK API, and neither has a changelog in what I was given. Before adopting, check three things: the Python SDK installation page for whether any local resolution exists, the REST API reference for rate limits and error semantics, and the current terms for the hosted service to see how the MIT grant maps onto it. If you cannot tolerate the external call, the answer is no, and no amount of query ergonomics changes that.

Editorial conclusion

Adopt AgentQL if your scraping targets change markup often, you are already writing Playwright scripts, and you accept that query resolution depends on an external service. Do not adopt it if you need a fully offline pipeline, if your data cannot leave your infrastructure, or if the pages you target are simple enough that a CSS selector or an XPath expression has been stable for years. Before committing, verify three things in the docs: whether the Python SDK ships a local query engine or only calls the REST API, what the API rate limits and failure responses look like, and whether the MIT licence on this repository covers the query language and the hosted endpoint or only the client code.

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. tinyfish-io/agentql on GitHub