sql-metadata answers what a SQL query touches, and where

The question the package answers: which columns does this query touch

sql-metadata is a Python package for reading and manipulating genomics-shaped data, in the sense that its data is SQL queries, and its job is to describe a query rather than run it.

The repository description says it uses a tokenized query returned by python-sqlparse and generates query metadata, and the README now says it uses sqlglot to parse SQL queries and extract metadata. The description is the older wording, and the README reflects the rewrite that version 3 brought, which is worth knowing before you rely on either.

The extracted metadata is: the columns a query uses, the tables it reads or writes, and the aliases in play. On top of that it resolves aliases automatically, so an alias such as a in a join is reported as the real column and table it refers to. It also provides a helper for normalization of SQL queries, and there is an interactive demo at sql-app.infocruncher.com for testing what it can do.

The audience is a developer who has a query as text and needs to know what is in it. That is a linter checking that a query only touches allowed columns, a data pipeline that maps a query to a set of upstream sources, a test that asserts a generated query references the right tables, or a tool that has to decide whether two queries are the same query after normalisation. It is not a query planner and it does not connect to a database.

The supported dialects are listed as MySQL, PostgreSQL, Sqlite, MSSQL and Apache Hive, with a caveat in the README that the listed backends can differ quite substantially but should work in regard of the query types supported by sql-metadata.

Version 3 swapped python-sqlparse for sqlglot

The release history tells the story in two lines. The v3.0.0 tag is titled v3 - sqlglot as the parsing library, published on 2026-06-30, and v3.0.1 followed the same day with a few fixes and updated dependencies. Before that, v2.20.0 on 2026-03-04 fixed an infinite loop in Parser.with_names and improved the handling of the source column in Parser.columns.

That is a major version bump for a library whose public surface is a single Parser class, and the reason is visible in the dependencies. The pyproject.toml declares only two runtime requirements:

[tool.poetry.dependencies]
python = "^3.10"
sqlglot = "^30.12.0"

One parser, pinned to a 30.x line of sqlglot, and Python 3.10 or newer. Nothing else.

The change matters because sqlglot is a transpiler with a full dialect system, not a tokeniser. python-sqlparse splits a query into tokens and leaves structure to the caller, which is why the older version of this library was described in terms of tokenized queries. sqlglot parses into an abstract syntax tree with resolved scopes, which is what makes the automatic alias resolution in the current version possible at all.

The dependency being pinned to a minor line also means that the parser's behaviour can change when sqlglot changes within that range, and a query that parsed yesterday may be reported differently today. The version line in the pyproject.toml is the constraint, and the sql_metadata/ package is where the metadata logic lives.

For an adopter, the practical advice is to pin sql-metadata exactly if you need stable behaviour, and to read the changelog when you bump, because the v2 to v3 transition is a change of parsing engine rather than a change of API.

Columns and output columns answer different questions

The most useful distinction in the API is between columns and output columns, and it is the one most people get wrong when they start.

The columns property gives you the real columns the query reads, with aliases resolved back to where they came from. So a query selecting a, then b plus c minus u as alias1, and custome_func(d) as alias2, yields a, b, c, u and d as its columns, and the aliases are exposed separately through columns_aliases_names and columns_aliases.

The output_columns property gives you the ordered list of names the SELECT would produce, preserving the aliases. The README makes the difference explicit: output_columns returns the ordered list of names that the SELECT would produce, preserving aliases, unlike columns, which resolves aliases back to real columns. It works with function calls, window functions and computed aliases, a SELECT star stays as the literal asterisk, and non-SELECT queries return an empty list.

from sql_metadata import Parser

# output_columns returns the ordered list of names that the SELECT would produce,
# preserving aliases (unlike `columns`, which resolves aliases back to real columns)
Parser("SELECT a, b AS c FROM t").output_columns
# ['a', 'c']

So for a query like a select of id, UPPER(email) as email_upper, and a ROW_NUMBER window function as rn, the output columns are id, email_upper and rn, while the columns property would give you the underlying email and id.

That distinction is the whole reason to use this package rather than a regular expression over the query text. A regex cannot tell you whether an identifier is a column, an alias or a table name, and it cannot resolve a to product_a.users.ip_address. The parser can, and it can do it for a query with comments, backticks and nested subqueries in it, all of which appear in the README examples.

Alias resolution is automatic, and columns_dict groups by clause

The README's join example is the clearest demonstration of what the package does beyond tokenising.

parser = Parser("SELECT a.* FROM product_a.users AS a JOIN product_b.users AS b ON a.ip_address = b.ip_address")

# note that aliases are auto-resolved
parser.columns
# ['product_a.users.*', 'product_a.users.ip_address', 'product_b.users.ip_address']

# note that you can also extract columns with their place in the query
# which will return dict with lists divided into select, where, order_by, group_by, join, insert and update
parser.columns_dict
# {'select': ['product_a.users.*'], 'join': ['product_a.users.ip_address', 'product_b.users.ip_address']}

Two behaviours in one example. The columns list has already resolved a and b to the tables they alias, so the output is fully qualified, which is what you want for a permission check. And columns_dict gives you the same information grouped by the clause it appeared in, across select, where, order_by, group_by, join, insert and update.

The clause grouping is what makes the tool useful for access control. Knowing that a column appears in the where clause is different from knowing it appears in the select list, and a tool that enforces column level permissions needs the former.

The alias side is exposed in the same way. tables_aliases gives a dictionary of alias to table, for example f to foo for a query selecting f.test from foo as f, and columns_aliases gives alias to the columns it refers to, for example alias1 to the list b, c and u. There is also columns_aliases_dict, which records which aliases were used in which section of the query, so the same alias can be tracked in the order by clause and in the select list separately.

The README notes that the columns list does not contain the aliases of the columns, which is the same distinction as columns versus output columns, applied to aliases.

Query type detection and how invalid SQL fails

Before you can route a query you often need to know what kind of query it is, and the package answers that with a str-enum.

from sql_metadata import Parser, QueryType

Parser("SELECT * FROM foo").query_type
# <QueryType.SELECT: 'SELECT'>

# QueryType is a str-enum, so it compares equal to both strings and enum values
Parser("INSERT INTO foo VALUES (1)").query_type == QueryType.INSERT   # True
Parser("INSERT INTO foo VALUES (1)").query_type == "INSERT"           # True

# REPLACE INTO is reported distinctly from INSERT
Parser("REPLACE INTO foo VALUES (1)").query_type
# <QueryType.REPLACE: 'REPLACE'>

# Supported types: SELECT, INSERT, REPLACE, UPDATE, DELETE,
# CREATE, ALTER, DROP, TRUNCATE, MERGE

Being a str-enum rather than a plain enum is a small design decision with a real payoff, because the value compares equal to the string form. That means a caller can compare against a literal without importing the enum, and a value read from a configuration file or a database row can be compared directly.

REPLACE INTO being reported separately from INSERT is the kind of detail that matters if you are classifying statements, since the two have different semantics even though both write rows.

Invalid input raises rather than returning an empty result, and the exception type is chosen so it does not break existing code:

from sql_metadata import Parser, InvalidQueryDefinition

# structurally invalid SQL raises `InvalidQueryDefinition` (a subclass of
# `ValueError`, so existing `except ValueError` handlers keep working)
try:
    Parser("").query_type
except InvalidQueryDefinition as exc:
    print(exc)  # "Empty queries are not supported!"

Subclassing ValueError is the compatibility detail. A caller that already catches ValueError around its SQL handling keeps working, and the error message is specific enough to tell an empty query apart from a query whose type is not supported.

Insert values are extracted as data, with placeholders when needed

The last extraction the README documents is the values of an insert, and it is the one that most obviously needs a parser rather than a split.

from sql_metadata import Parser

parser = Parser(
    "INSERT /* VoteHelper::addVote xxx */  INTO `page_vote` (article_id,user_id,`time`) " 
    "VALUES ('442001','27574631','20180228130846')"
)
# extract values from query
parser.values
# ["442001", "27574631", "20180228130846"]

The example is deliberately awkward, with a comment inside the statement, backticked identifiers and a mix of types, and the package returns the values as a list of strings in column order. There is a values_dict as well, which pairs the values with the column names, so a row of data comes back as a mapping rather than a positional list.

When the statement does not name its columns, the package invents placeholders. The README shows an INSERT IGNORE with three values and no column list, and the values_dict comes back with column_1, column_2 and so on. That is a sensible convention for round tripping data, and it is also the place where you should look first if a value looks like it landed in the wrong place.

Taken together with the columns and tables extraction, the package covers the three things a tool usually needs from a query: what it reads, what it writes, and what it does. What it does not give you is the meaning. The package reports names and structure, so a query that filters on the wrong column is not something it can tell you about.

The toolchain demands full coverage and strict typing

The development configuration is the most demanding part of the project, and it is a good signal about how the maintainer works.

Coverage is enforced at 100 percent:

[tool.coverage.report]
show_missing = true
fail_under = 100

The Makefile has a coverage target that runs pytest with the coverage plugin and writes both a terminal report and an HTML report, and a separate test target that runs pytest in verbose mode. Publishing goes through poetry build and poetry publish, with a comment reminding you to tag the release first.

Linting and formatting are ruff, with a line length of 88 and a target version of py310, and a rule selection of E, F, W, C90 and I, which is errors, pyflakes, warnings, complexity and import sorting. The complexity limit is explicit:

[tool.ruff.lint.mccabe]
max-complexity = 8

A complexity ceiling of 8 is a meaningful constraint for a parser, because a parser is exactly the kind of code that grows a decision tree. The mccabe rule is there to stop that.

Typing is checked with mypy, and the settings are strict: check_untyped_defs, disallow_untyped_defs and disallow_any_generics are all on, and warn_return_any is on, so a function that is annotated but returns an untyped Any is reported. The python_version is pinned to 3.10 to match the runtime requirement, and ignore_missing_imports is on so that a missing stub for a third party package does not fail the run.

The repository root also holds AGENTS.md, CLAUDE.md and ARCHITECTURE.md, so the project documents both its architecture and how to work on it with an assistant. The tests live in a test/ directory, and the README points at specific test files, tests/test_getting_columns.py and tests/test_getting_tables.py, for more examples of the behaviour.

Where it is the wrong tool, and how the release line behaves

There are three boundaries worth stating plainly.

The first is that this is a name resolver, not an interpreter. It tells you which columns and tables a query mentions and where they appear. It does not evaluate the query, does not know whether a predicate is sensible, and does not tell you that a join is wrong. If your problem is understanding behaviour rather than enumerating references, this is the wrong tool.

The second is dialect coverage. The README names MySQL, PostgreSQL, Sqlite, MSSQL and Apache Hive, and then warns that the listed backends can differ quite substantially but should work in regard of the query types supported by sql-metadata. The backing store is sqlglot, which supports many more dialects than those five, but the README is careful to claim only what it has tested, and a dialect-specific construct outside the supported set is where you will find surprises.

The third is the version line. Version 3 changed the parsing engine, and the v2 line ended at 2.20.0 with a fix for an infinite loop in Parser.with_names and better handling of the source column in Parser.columns. That bug history is a reminder that this is a parser of untrusted text, and untrusted text is where parsers loop or blow the stack.

The repository is not archived, and the last push was on 2026-10-19. The package is on PyPI as sql-metadata, the homepage is the PyPI page, and the licence is MIT, with the LICENSE file at the root and the same identifier in the pyproject.toml.

Editorial conclusion

Adopt sql-metadata if you need to inspect SQL programmatically, for a linter, a data pipeline guard, a column level access check or a test that asserts which columns a query touches, and if you want alias resolution rather than raw tokens. Do not adopt it if you need to understand what a query means, because it reports names and structure, not semantics, and it does not execute or plan anything. Before you depend on it, pin the version, because version 3 changed the parsing library and the release notes describe the v2 line as ending at 2.20.0, and check that the dialect you use is one of the listed backends, since the README warns that the listed backends can differ substantially in the query types they support.

Frequently asked questions

Official sources

1. License: MIT
2. macbre/sql-metadata on GitHub
3. Project website
4. README
5. Releases