# SQLLineage for table and column lineage analysis > SQLLineage parses SQL to identify source and target tables, trace column dependencies, and combine lineage across multiple statements. It suits engineers working with data discovery and governance who need a command-line tool rather than a separate metadata service. **reata/sqllineage** — SQL Lineage Analysis Tool powered by Python - Repository: https://github.com/reata/sqllineage - Stars: 1,684 · Forks: 285 - Language: Python - License: MIT - Published: 2026-10-16 · Updated: 2026-10-16 · Language: en - Canonical page: https://hysenlabs.com/projects/reata-sqllineage ## SQLLineage's table and column lineage scope SQLLineage addresses a specific analytical problem: determining where SQL data comes from and where it goes without requiring the caller to interpret parser tokens, keywords, or identifiers. It is aimed at engineers handling SQL scripts, data discovery, metadata collection, and data governance. The primary output identifies source tables and target tables, while an optional column-level mode prints lineage paths for individual output columns. The tool can analyze a quoted query or a SQL file. That makes the same package useful for a quick check during development and for processing a saved script, although the README focuses on command-line usage rather than documenting a stable Python API. SQLLineage also recognizes intermediate tables when it combines several statements. This matters for chained transformations because an intermediate table may not be the original source or final destination of the complete script. It is not a database profiler. Its documented purpose is static analysis of SQL text and any metadata supplied with it. It does not document query execution plans, query results, or runtime data movement. Its output should therefore be understood as declared lineage derived from SQL structure, not proof that a statement ran successfully. ## Parser, graph, and lineage data flow SQLLineage uses pluggable parser libraries, with `sqlfluff` and `sqlparse` identified in the README. The parser turns SQL into an abstract syntax tree, which SQLLineage analyzes to locate reads, writes, common table expressions, drops, renames, and column relationships. The project then stores lineage in a graph maintained by a pluggable graph operator. Two interchangeable graph implementations are documented. The default implementation is based on `networkx`, while `rustworkx` is an opt-in alternative. Both are listed as project dependencies in `pyproject.toml`. The repository also includes a `sqllineagejs` directory and a Dockerfile based on a Python and Node.js image, but the README centers the analysis flow on Python SQL parsing and graph representation. For multiple statements, the tool combines their results. In the README example, one statement writes `db1.table1` and another reads that table while writing `db3.table3`. The combined output treats `db2.table2` as the source, `db3.table3` as the target, and `db1.table1` as an intermediate table. Verbose mode exposes the per-statement reads and writes before printing that summary. ## Installing SQLLineage and parsing the first query The README directs users to install the released package from PyPI. `pyproject.toml` specifies Python 3.10 or later and declares the `sqllineage` console entry point. ```bash $ pip install sqllineage ``` After installation, the first documented use parses an `INSERT ... SELECT` statement passed with `-e`. The expected output contains one statement, one source table, and one target table. ```bash $ sqllineage -e "insert into db1.table1 select * from db2.table2" ``` For a saved query, replace the inline statement with `-f` and the file path. The README calls this file `foo.sql` and reports its tables as `db1.table_foo`, `db1.table_bar`, and `db2.table_baz`. ```bash $ sqllineage -f foo.sql ``` The repository also defines a Dockerfile that installs the local project and starts `sqllineage` as the non-root `sqllineage` user. Its default command serves the interface through `-g`, binds to `0.0.0.0`, and takes the port from `$PORT`. The Dockerfile does not include a complete `docker run` example, so the container command and network behavior should be checked against the project documentation before deployment. ## Dialect selection for non-ANSI SQL SQLLineage uses the `ansi` dialect by default. The README warns that valid warehouse-specific syntax can fail under ANSI rules, and that a reserved keyword in one dialect may be accepted as an identifier in another. Dialect selection is therefore not a cosmetic setting for a mixed SQL estate. The documented example uses `INSERT OVERWRITE TABLE map SELECT * FROM foo`. ANSI and Hive parsing raise `sqllineage.exceptions.InvalidSyntaxException`, while `sparksql` identifies `.foo` as the source and `.map` as the target. ```bash $ sqllineage -e "INSERT OVERWRITE TABLE map SELECT * FROM foo" --dialect=sparksql ``` The exact list of supported dialect names is delegated to the command rather than duplicated in the README. ```bash $ sqllineage --dialects ``` This limitation is visible rather than hidden. A successful parse under one dialect does not prove compatibility with another warehouse parser, and selecting the wrong dialect can misclassify a table name as a syntax error. SQLLineage is a good fit when dialect names correspond to the SQL actually being analyzed. It is the wrong tool for statements containing syntax that no available parser understands. ## Column-level paths and metadata limits The `-l column` option asks the CLI to print column lineage paths. In the README example, the tool follows columns through joins, computed expressions, aggregates, and a second transformation. A path such as `.corge.col1 <- .foo.col1 <- .bar.col1` shows the dependency chain rather than only the two tables involved. The output also exposes two metadata boundaries. A wildcard such as `d.*` remains as `.foo.* <- .quux.*`; SQLLineage does not know the columns encoded by `*` unless metadata is supplied. An unqualified input can remain `.foo.col4 <- col4` when the available context does not identify whether `col4` comes from `bar`, `baz`, or `quux`. The README explicitly describes this as the best result possible without metadata. Optional metadata can improve the output when table schemas are known. The README introduces a SQLite database named `db.db` and says tables are created there, but the visible documentation does not show the complete command for loading that metadata. Consequently, the exact metadata-loading syntax cannot be confirmed from the README. Without a schema, treat column lineage as structurally derived rather than fully resolved. ```bash $ sqllineage -f test.sql -l column ``` This command prints dependency paths for the SQL stored in `test.sql`. It does not expand unresolved wildcards by itself. ## SQLLineage compared with database catalog inspection A real alternative is to derive lineage from the database catalog and workload metadata maintained by the database platform. That approach can connect SQL lineage to runtime observations, permissions, freshness, and table definitions that are unavailable in a standalone SQL file. It also requires access to the database environment and its native logs or audit facilities. SQLLineage takes a different approach. It parses SQL text, selects a dialect, analyzes the resulting syntax structure, and builds lineage in memory through a graph implementation. Its advantage is portability across inputs that can be represented as SQL and graph analysis without a live database connection. The trade-off is that parser compatibility and available metadata determine the completeness of the result. The choice depends on the required evidence. SQLLineage is suitable when the immediate need is source-to-target or column dependency analysis for a known script. Catalog inspection is more suitable when the project must establish which statements actually executed against production tables. These approaches can also be complementary, using SQLLineage for static scripts and catalog records for observed execution, but the README does not document such an integration. ## Release cadence, packaging, and upgrade review The repository is not archived, and the last push was on 2026-10-16. Recent releases include `v1.5.9` on 2026-09-05, `v1.5.8` on 2026-05-16, and `v1.5.7` on 2026-01-11. These dates show recent repository activity and released versions, but the README does not define a long-term support window or a compatibility policy for upgrading between releases. Upgrade work includes reviewing parser output for warehouse-specific SQL, rerunning representative lineage checks, and reviewing changes to the `sqlfluff`, `sqlparse`, `networkx`, or `rustworkx` dependencies selected in the environment. `pyproject.toml` permits Python versions from 3.10 through 3.14 and lists an operating-system-independent classifier, while the Dockerfile uses Python 3.10 with Node.js 18. The repository declares the MIT license. That identifies the license granted for the code, but adopting it also requires checking how MIT interacts with the licenses of separately distributed parser, graph, and container dependencies. No legal interpretation is needed to understand the adoption boundary: distribution terms apply to more than the top-level project license alone. ## Conclusion Engineers who need local SQL table or column lineage, especially across multi-statement scripts, should adopt SQLLineage when dialect-aware parsing is acceptable. Data platforms that require complete wildcard expansion, resolved metadata, or an undocumented storage catalog should verify those cases first. For the documented CLI workflow, start by confirming that `sqllineage` identifies the intended source and target tables under the correct `--dialect`. ## FAQ ### How do I install SQLLineage and analyze a SQL file? Install the package with `pip install sqllineage`, then run `sqllineage -f foo.sql`. The command reports source, target, and, where applicable, intermediate tables. ### How do I analyze column-level lineage in SQLLineage? Use `sqllineage -f test.sql -l column` for a file or use the `-l column` option with `-e`. Unresolved wildcards and unqualified columns remain limited when metadata is unavailable. ### What should I check when SQLLineage reports an InvalidSyntaxException? Check the selected dialect because SQLLineage uses `ansi` by default. The README demonstrates that the same `INSERT OVERWRITE` statement fails under ANSI and Hive but succeeds under SparkSQL. ## Sources - [Issues](https://github.com/reata/sqllineage/issues) - [License: MIT](https://github.com/reata/sqllineage/blob/master/LICENSE) - [README](https://github.com/reata/sqllineage/blob/master/README.md) - [reata/sqllineage on GitHub](https://github.com/reata/sqllineage) - [Releases](https://github.com/reata/sqllineage/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/reata-sqllineage