Hercules: a Gherkin-driven testing agent from TestZeus

The maintenance problem Hercules is aimed at

Selector-based UI suites rot. A class name changes, a modal gains a wrapper div, and a test that passed yesterday fails for reasons unrelated to the product. The README frames Hercules as an answer to that: it turns "simple, easy-to-write Gherkin steps into fully automated end to end tests" and claims no coding skills are needed. The intended user is a QA engineer or product person who can write a feature file in business language but does not want to own a page-object layer, plus teams that want the same feature files running in a CI/CD pipeline. The project also lists API, security, accessibility and visual validation as in scope, so the pitch is one runner for several kinds of check rather than a browser-only tool. The repository is Python, licensed AGPL-3.0, and the release history shows 1.0.0 in July 2026, then 1.0.1 and 1.0.2 in the following weeks. That cadence suggests a project that has only recently declared a stable line, which matters when you decide how tightly to pin a version.

How the agent executes a Gherkin step

The README does not publish a full architecture diagram, but the pieces it does show are enough to describe the loop. You supply a feature file. Hercules reads the Gherkin steps and, rather than mapping each step to a hand-written function, uses an agent that decides what to do in the browser through Playwright, which is why the install instructions include a separate Playwright browser download. The repository topics name autogen and "large-action-model", so the step resolution is model-driven rather than a fixed step-definition registry. Results come back as a report; the README shows an SVG artefact under statics/assets/hercules.svg as the output illustration. The Python sandbox is the escape hatch when the agent should not improvise. A step such as "execute the apply_filter function from script at scripts/apply_filter.py with filter_type as Turtle Neck" loads your own async function, and inside it the globals page, browser, logger and asyncio are injected automatically. The example function tries two selectors in a loop and clicks the first that matches. That is the real dividing line in this design: prose steps for the parts where the agent can reason about intent, and plain Playwright code for the parts where you want an exact selector. The docs directory includes docs/python_sandbox_execution.md, docs/Migration/MIGRATION.md and docs/Migration/ARCHITECTURE.md, so the maintainers treat the runtime behaviour as something worth writing down.

Install, run, and the sandbox tenant switch

The PyPI route is two commands. First pip install testzeus-hercules, then playwright install --with-deps to fetch the browser binaries and their system libraries. The CLI accepts an input file, shown in the README as testzeus-hercules --sandbox-tenant-id executor_agent --input-file test.feature. The sandbox tenant is the security dial on script execution. Setting SANDBOX_TENANT_ID="executor_agent" in the environment, or passing the same value on the command line, grants the full module set the README lists: requests, pandas, numpy and BeautifulSoup. The README describes four modes in total (executor, data, API and restricted) but only spells out the executor tenant in the example, so the exact module allowlist for the other three is something you will need to read in the sandbox documentation rather than infer. There is also a Colab notebook linked for a first look, with a caveat that the session may need a restart after installing dependencies. A Docker image exists under testzeus/hercules, which is the more sensible path for CI because it avoids installing Playwright's OS-level dependencies on every runner.

Where the agent approach breaks down

A model-driven runner is not deterministic in the way a step-definition suite is. The same feature file can take a different path to the same assertion, and when it fails you get a narrative rather than a stack trace pointing at line 42 of a helper. For a smoke suite that is acceptable. For a regulated release gate where you must reproduce a failure exactly, it is a poor fit. The second constraint is cost and latency: every step is a model call, so a long end-to-end journey is slower and more expensive than a headless Playwright script doing the same clicks, and the bill scales with how often CI runs. Third, the agent depends on the page being reachable and the model endpoint being available; a network partition takes the whole suite down, not one test. Fourth, the sandbox tenant modes exist precisely because executing arbitrary Python from a test file is dangerous, and the README's own default example hands the script requests and BeautifulSoup. If your tests run against production data, that combination deserves a deliberate decision rather than a copy-paste of the example environment variable.

Hercules against plain Playwright and pytest

The obvious alternative is Playwright driven by pytest, which is what most Python teams reach for. The difference is where the intent lives. In pytest you write the selector and the assertion; the test is code, and it fails loudly at a known line. Hercules moves the intent into Gherkin and lets the agent choose the selector at run time, which is what buys resilience to markup changes and what costs you determinism and a per-step model call. The trade is not free in either direction: a pytest suite is cheap and reproducible but needs a human every time the DOM shifts, while a Hercules feature file is cheap to write and expensive to run. There is a middle position the project itself supports, which is to keep the brittle interactions in the Python sandbox where you control the selector and let the agent handle the exploratory parts. If your team has no Gherkin habit and no appetite for model spend, plain Playwright is the smaller commitment. If your bottleneck is maintaining selectors across a large, frequently redesigned application, the agent loop addresses the actual pain.

Licence and the cost of staying current

Hercules is AGPL-3.0. That is a copyleft licence with a network clause, and it is a different proposition from MIT or Apache-2.0 for a tool that sits inside a company's delivery pipeline. Running it internally to test your own application is one thing; wrapping it in a hosted service you expose to customers is another, and the AGPL's source-availability obligation is the part to read carefully. This is not legal advice, and the specific answer depends on how you deploy it, so route the question to whoever handles licensing at your organisation before the tool becomes load-bearing. On maintenance, the visible signals are the 1.0.0 to 1.0.2 releases within roughly three weeks, a docs/Migration directory that includes both MIGRATION.md and ARCHITECTURE.md, and a CI workflow at .github/workflows/main-test.yml. The migration docs imply the runtime behaviour has changed enough to warrant a written upgrade path, which means pinning an exact version in your requirements file is worth the small effort. The upgrade cost is not just the package: a new Hercules version can change how the agent interprets a step, so your feature files are part of the upgrade surface, and the migration guide is the document that tells you which steps to re-check.

Who should pick this up, and what to check first

The fit is a team that already writes Gherkin, tests a web application whose markup changes often, and is comfortable paying for model calls on every CI run. The misfit is a team that needs bit-for-bit reproducible runs, tests in an air-gapped environment, or cannot send application data to an external model provider. Before you commit, install the pinned version, run a single existing feature file through testzeus-hercules --input-file, and confirm the report matches what the README shows. Then read docs/run_guide.md for the project layout and docs/Migration/ARCHITECTURE.md for how the runtime is put together, because those two documents describe behaviour the README only gestures at. Finally, decide your sandbox tenant deliberately: the executor tenant in the README example grants requests, pandas, numpy and BeautifulSoup to scripts loaded from your feature files, and that is a broader grant than most test suites need. Start with the restricted mode, widen it only for the scripts that require it, and treat the tenant id as a security control rather than a convenience flag.

Editorial conclusion

Adopt Hercules if your team already writes Gherkin and wants an agent to drive Playwright without maintaining page objects, and if you can live with AGPL-3.0 and per-run model cost. Do not adopt it if you need deterministic, byte-identical test runs or cannot send test data to an external model provider. Before committing, run one existing feature file through the CLI, pin the exact 1.0.x version you install, and check docs/run_guide.md plus docs/Migration/ARCHITECTURE.md against your CI runner's memory and browser dependencies.

Official sources

1. License: AGPL-3.0
2. Project website
3. README
4. Releases
5. test-zeus-ai/testzeus-hercules on GitHub