Schemathesis: schema-driven fuzzing for OpenAPI and GraphQL APIs

The API bugs Schemathesis is built to find

The README lists five failure classes it targets: 500 errors on edge case inputs, responses that do not match the documented schema, validation bypasses where invalid data is accepted, integration failures where responses do not match client expectations, and stateful bugs where operations pass individually but fail in a realistic workflow. The last one is the interesting one. Most schema-driven tools test one endpoint at a time. Schemathesis infers operation links from the schema and runs them in sequence, which is how it reaches bugs that only appear after a create followed by a read.

The audience is API teams that already keep a schema as part of their contract. If the schema is accurate, the tool has something to work from. If it is stale or hand-waved, the generated tests inherit that weakness, and the findings will be noise. The README's own framing is that this is for catching bugs before users do, not for replacing unit tests around business logic that the schema cannot express.

How schemathesis turns a schema into test cases

The mechanism is property-based testing. Schemathesis is built on top of Hypothesis, a Python property-based testing library, as the acknowledgements section states. Instead of a fixed list of cases, it derives inputs from the constraints in your OpenAPI or GraphQL document, then shrinks failing inputs to a minimal reproduction.

Two layers sit on top of that generation. The adaptive layer learns constraints, ids and auth values from server responses and reuses them mid-run, so later requests are built from data the API actually returned rather than from guesses. The stateful layer builds a state machine from operation links inferred from the schema, and the README gives the shape of that workflow: create user, get user, delete user. You can also feed a fuzz dictionary, mixing real ids, wordlists or LLM-generated payloads into the generated data, which matters when random strings never satisfy a server-side lookup.

Generation is only half of it. Built-in checks validate responses against the schema, and you can add custom validation checks for your own business rules alongside them. A keyword-level coverage report shows which schema constraints your tests actually exercised, which is the honest way to see whether a run was thin or thorough.

Installing Schemathesis and running a first test

The README offers a zero-install path through uvx, which downloads and runs the CLI in one step. The demo schema it points at is a public example API, so you can see output before pointing the tool at anything of your own.

uvx schemathesis run https://example.schemathesis.io/openapi.json

You should see the CLI walk the operations in that schema and report any failures it finds. To install it properly into an environment, the README gives the pip-style command through uv, followed by a run against your own schema URL.

uv pip install schemathesis
schemathesis run https://your-api.com/openapi.json

If you would rather not write Python, the configuration file `schemathesis.toml` carries auth, phases and per-operation overrides. The README's example sets a bearer token from an environment variable, raises the generation ceiling to 500 examples and lets the rate limiter follow `Retry-After` headers on 429 responses.

headers = { Authorization = "Bearer ${API_TOKEN}" }
generation.max-examples = 500
rate-limit = "auto"

For teams that live in pytest, the integration is a decorator on a test function. The schema is loaded from a URL, `parametrize` turns each operation into a test case, and `call_and_validate` sends the request and checks the response in one call.

import schemathesis

schema = schemathesis.openapi.from_url("https://your-api.com/openapi.json")


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

Stateful testing reuses the same schema object. Calling `as_state_machine` produces a test class the README says works with pytest or unittest.

APIWorkflow = schema.as_state_machine()
TestAPI = APIWorkflow.TestCase

In CI, the repository ships a GitHub Action. The README's snippet passes the schema URL as an input.

- uses: schemathesis/action@v3
  with:
    schema: "https://your-api.com/openapi.json"

Where Schemathesis is the wrong tool

Stateful testing is the sharpest edge. The README's own example is create user, get user, delete user, which means a run against a live environment creates and removes real records. There is no statement in the README that the tool isolates or rolls back what it writes. Point it at production and you are asking for data damage. The rate limit setting exists precisely because a fuzzing run is a load generator, and `auto` exists because a 429 can otherwise turn into a cascade of failed assertions.

The second limit is the schema itself. GraphQL support depends on `hypothesis_graphql`, and everything about generation quality depends on how completely the document describes the API. An OpenAPI file with loose types and no constraints gives the generator little to work with, and the coverage report will show it.

The third is the Python 3.10 floor stated in `pyproject.toml`. Teams on older interpreters cannot install the current release. And the README carries an upgrade warning pointing at MIGRATION.md, which is a signal that the 4.x line changed things in ways that break existing test code. Budget time for that read before upgrading a working suite.

Schemathesis compared with Pact

The natural comparison is contract testing, and Pact is the name that comes up. The difference is in what each one treats as the contract.

Pact is consumer-driven: consumers declare the interactions they expect, and the provider is verified against those recorded expectations. The contract is a byproduct of client code. Schemathesis runs the other direction. The schema is the contract, and the tool generates requests that probe whether the server honours it, including inputs no consumer would ever send. That is why it finds 500s and validation bypasses: it is deliberately trying inputs outside the happy path.

In practice these are complements. Pact tells you whether the provider still satisfies what known clients need. Schemathesis tells you whether the provider holds up under inputs nobody wrote down. The README's own framing of the stateful mode, operations that pass alone but fail in a workflow, is closer to what Pact's provider verification does, but Schemathesis derives those sequences from the schema rather than from recorded consumer interactions.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-09-22, with v4.28.0 released the same day. That is a fast release cadence, and it cuts both ways: fixes arrive quickly, but so do changes that affect existing suites. MIGRATION.md at the repository root exists for that reason, and the README flags it explicitly for anyone upgrading from older versions.

The licence is MIT, stated in both the README and the `license` field of `pyproject.toml`. MIT is permissive: you can use it commercially, modify it and redistribute it, provided the copyright notice and permission notice are kept. That is a description of the licence text, not legal advice, and if you vendor or redistribute it inside a product, have your own counsel read the terms.

The upgrade cost is mostly yours to manage. The dependency list pins Hypothesis to a range, and Hypothesis itself evolves, so a Schemathesis upgrade can pull in behavioural changes in generation. The replay and baseline features exist to soften this: you can re-run past failures and fail CI only on new ones, which is the practical way to absorb a version bump without drowning in previously known failures.

Editorial conclusion

Adopt Schemathesis if you have an OpenAPI or GraphQL schema and a Python test stack, and you want generated inputs, response-driven adaptation and stateful workflows without hand-writing cases. Do not adopt it if your API has no machine-readable contract, or if you cannot give it a disposable environment, because stateful runs create, modify and delete real resources. Before committing, read MIGRATION.md for the 4.x changes, decide whether the TOML config or the pytest integration is your primary interface, and confirm how you will store and replay failures.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. schemathesis/schemathesis on GitHub