SQLancer: Automated Database Logic and Performance Bug Finding

SQLancer's Database Testing Boundary

SQLancer targets the DBMS implementation, not the correctness of queries written by an application user. The README makes this distinction explicit: the tool is intended to find bugs in the code of the database system itself. That makes it different from a query validator or a normal integration test suite, where expected rows are often defined by a product requirement.

The intended users are teams working on database engines, including developers who need exploratory testing for logic errors, crashes, internal failures, and selected performance issues. Researchers can also use its generators and test oracles when studying automated DBMS testing. SQLancer is less appropriate for teams that only need to prove that a known query returns a known result. In that case, a normal database test fixture is usually a clearer choice.

The project claims that it has found hundreds of bugs in mature and widely known DBMSs. That is a project-level claim in the README, not a measurement produced by this article. The documented coverage is broad enough to include databases such as SQLite, DuckDB, H2, MySQL, MariaDB, PostgreSQL, CockroachDB, and TiDB, although the exact configuration and oracle availability depend on the selected DBMS implementation.

SQL Generation, Test Oracles, and Database State

SQLancer follows a multi-phase input-generation process. It first creates a database schema consisting of tables and columns. It then inserts data and constructs additional database states, including indexes, views, and database-specific options. Finally, it generates queries and passes those queries to a result validator, also called a test oracle.

The important design choice is that generated SQL is not treated as a self-validating answer key. Instead, SQLancer compares results through mechanisms designed for particular bug classes. The README lists Ternary Logic Partitioning, NoREC, Pivoted Query Synthesis, Differential Query Plans, and CODDTest as complementary oracles. It also describes CERT for identifying cases where a DBMS could reasonably have produced a result more efficiently by using cardinality estimation restriction testing.

Input generation can be unguided, as in the standard approach, or feedback-guided. The documentation says the feedback-guided approach aims to exercise as many unique query plans as possible, based on the idea that varied plans may expose different database behaviours. SQLancer also detects unexpected internal errors by allowing a database-specific list of potential errors for a query. Crashes use an implicit oracle. These mechanisms give the tool several ways to interpret a disagreement or failure, but they also mean that a detected issue needs to be reproduced and understood rather than treated as a generic test verdict.

First SQLite3 Test with NoREC

The README gives a first run against SQLite3 using the NoREC oracle. SQLancer requires Java 11 or above and Maven. The documented commands clone the repository, build a JAR, enter the target directory, and start four testing threads against SQLite3.

The following commands build SQLancer and start the documented SQLite3 test. The last command uses four threads and selects NoREC after the DBMS name, because database-specific options must follow the DBMS identifier.

git clone https://github.com/sqlancer/sqlancer
cd sqlancer
mvn package -DskipTests
cd target
java -jar sqlancer-*.jar --num-threads 4 sqlite3 --oracle NoREC

For this command, the README says progress information should be printed every five seconds. If no bugs are found, SQLancer continues running. Press CTRL+C to terminate it manually. To stop after a defined number of bugs, use `--num-tries`. To limit execution time, use `--timeout-seconds`.

General options such as `--num-threads` must appear before the DBMS name. DBMS-specific options, including SQLite's `--test-rtree` example or the `--oracle NoREC` value, appear after `sqlite3`. Running SQLancer with no parameters displays the available options and commands.

Logs, Reproduction, and Reduction Limits

SQLancer stores logs under `target/logs`. By default, `--log-each-select` is enabled, so every SQL statement sent to the DBMS is written to files whose names end in `-cur.log`. When SQLancer identifies a logic bug, it creates a `.log` file containing the statements needed to reproduce the issue. The documentation says this reproduction log includes the last executed query together with the statements that establish the relevant database state.

A generated failure is not always minimal. SQLancer may produce a sequence containing redundant statements, and the README says developers often need to reduce the input before analysis. Manual reduction removes statements or features one at a time, replays the case, and checks that the oracle still fails. For TLP and NoREC, that means confirming that the two relevant queries still produce different results.

SQLancer includes an experimental delta-debugging reducer enabled with `--use-reducer`. The README also mentions C-Reduce as an alternative, but C-Reduce requires a script that specifies the test oracle. This reducer is explicitly experimental, so its output should be checked against the original reproduction log. The README does not quantify reduction time, reduction success, or the reducer's behavior across every DBMS oracle.

Embedded Databases and External DBMS Setup

SQLite does not require a separate installation for the documented getting-started run because it is an embedded DBMS that runs in the same process as the application. The README lists DuckDB, H2, and SQLite as embedded systems supported through JAR dependencies. That setup simplifies the first run, but it also creates an operational limitation: a crash in one of these embedded systems can crash the JVM running SQLancer.

The repository includes a `Dockerfile` based on `ubuntu:21.04`. It installs `openjdk-15-jdk` and Maven, copies an already built `target/sqlancer-*.jar` as `sqlancer.jar`, copies `target/lib/*.jar` into `/lib/`, and defines a Java entrypoint. Because the Dockerfile assumes the project has already been built, it is not a substitute for the documented Maven build step. The provided Dockerfile does not document a database server, a port, or environment variables for connecting to an external DBMS.

The README's DBMS list is broader than the getting-started example. It identifies SQLite, DuckDB, H2, MySQL, MariaDB, PostgreSQL, CockroachDB, and TiDB as topics, but the documented example does not provide connection arguments or setup commands for the external systems. Users should therefore check the repository configuration for the specific DBMS before assuming that a database server can be tested from the same command.

When SQLancer Is the Wrong Tool

SQLancer is a poor fit for validating a fixed application query or verifying a business rule. It does not know the intended application semantics, and its purpose is to explore DBMS behaviour through generated schemas, states, and SQL. If the expected answer is already known and deterministic, an ordinary SQL test may be simpler to maintain.

It can also be a poor fit for a team that cannot tolerate long-running tests. The documented default behaviour is to execute indefinitely when no bug is found. `--num-tries` or `--timeout-seconds` can impose limits, but the README does not state how long a meaningful run should take for a particular DBMS or workload. Findings may require manual log analysis and oracle-specific reduction.

Compared with fixed regression tests, SQLancer provides broader generation and several specialised oracles, but its results are exploratory rather than a complete proof of database correctness. Compared with a manually maintained test suite, it can exercise database states that are tedious to enumerate, at the cost of interpreting unexpected outputs and controlling test duration.

MIT Licence, Releases, and Maintenance Cost

SQLancer uses the MIT licence. That identifies the project's licensing basis, but it does not answer questions about a particular DBMS, dependency, or deployment environment. Users reviewing an adoption should check the licence text and dependency terms separately if those matter to their organisation. No legal interpretation of the MIT licence is required to use this description.

The listed releases are `v2.0.0` on 2022-01-12, `1.1.0` on 2021-01-24, and `v1.0.1` on 2021-01-22. The repository is not archived, and the last push was on 2026-10-14. The listed release entries do not document the upgrade process, compatibility policy, or migration notes. Upgrading therefore requires reviewing project changes and testing the selected DBMS integration rather than assuming that all combinations remain unchanged.

The practical maintenance cost comes from DBMS-specific configuration, Java and Maven compatibility, log interpretation, and reproduction work. The MIT licence does not remove those operational tasks. The Dockerfile also assumes a prebuilt JAR, so container-based use requires a separate build process.

Editorial conclusion

SQLancer suits DBMS developers and researchers who can configure a target database and interpret generated logs and failing cases. It is a poor fit for checking one application query, and users should first verify Java 11 or above, Maven, the selected DBMS implementation, oracle support, and whether the experimental reducer meets their debugging workflow.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. sqlancer/sqlancer on GitHub