tree-sitter-python: the Python grammar that most tree-sitter tooling actually loads

A grammar is the layer everything else in tree-sitter depends on

tree-sitter-python is the Python grammar for tree-sitter, and the repository is MIT licensed with the version at 0.25.0. The framing matters, because a grammar is not a tool you run. It is a specification of the Python syntax, expressed in a form that tree-sitter can compile into an incremental parser, and every other piece of tree-sitter tooling for Python, from editor syntax highlighting to linters, depends on this file being present and correct. The README lists the two references the grammar is derived from, the Python 2 Grammar and the Python 3 Grammar documents, and the repository carries a grammar.js at the root, which is the human-editable source. The published artefacts are the generated parser and the language bindings, and the badges show the distribution channels, crates.io for Rust, npm for Node, and PyPI for Python.

The grammar source is JavaScript, the output is C

The build pipeline is the part that surprises people. The grammar is written in grammar.js, a JavaScript file, and the tree-sitter-cli compiles it into a C parser under src/, with the tree-sitter headers included in the package. The npm scripts show the loop:

    "lint": "eslint grammar.js",
    "prestart": "tree-sitter build --wasm",
    "start": "tree-sitter playground",
    "test": "node --test bindings/node/*_test.js"

So lint runs ESLint against grammar.js with the eslint-config-treesitter config, prestart produces a WebAssembly build, start launches the tree-sitter playground, the interactive editor for exercising a grammar against sample code, and test runs the Node binding tests. So the source of truth is JavaScript, the artefact everyone links against is C, and the playground is how you check that a grammar change did not break the tree.

Five language bindings, one parser

The repository ships bindings for the major tree-sitter host languages, and the top level listing shows them as a bindings/ directory alongside binding.gyp for Node, Package.swift for Swift, go.mod and go.sum for Go, Cargo.toml for Rust, and pyproject.toml with setup.py for Python. The npm package points its main and types at bindings/node and depends on node-addon-api and node-gyp-build, so the Node consumer gets a prebuilt native addon when one is available and compiles it otherwise. The Cargo package declares a single dependency, tree-sitter-language at 0.1, and uses cc as a build dependency to compile the C parser. The Python side is the most elaborate, and the setup.py shows why. It defines custom command classes that copy the queries directory into the built package, compile the extension with -std=c11 and hidden visibility on non-MSVC compilers, append the external scanner source if src/scanner.c exists, and set the limited API macro when the build is stable ABI.

Queries ship with the package, which is what makes them reusable

The queries/ directory holds tree-sitter query files, and every packaging path takes care to include them. The npm files array lists queries/*, the Cargo include list has queries/*, and the Python setup.py copies the queries tree into the built package and registers *.scm files as package data, with the EggInfo command including them recursively. The Python package also declares *.pyi and py.typed as package data, which is why the classifier list includes Typing :: Typed. This is the mechanism by which a highlight query, a locals query, or an injection query can be reused across every tool that loads this grammar, rather than each editor writing its own. The queries directory being a first-class citizen in all three build systems is the clearest signal of how the project is meant to be used.

The test corpus is about Python's messy edges

The examples/ directory is the test corpus, and the filenames are more informative than a list of features. There is examples/python2-grammar.py and examples/python3-grammar.py, each with a CRLF variant in examples/python2-grammar-crlf.py and examples/python3-grammar-crlf.py, and a examples/python3.8_grammar.py for a specific language version. Then there are the awkward cases. examples/mixed-spaces-tabs.py, examples/tabs.py, examples/trailing-whitespace.py, examples/multiple-newlines.py, and examples/compound-statement-without-trailing-newline.py cover the whitespace and newline behaviour that trips up naive parsers. examples/simple-statements-without-trailing-newline.py covers the same issue for simple statements. That is the shape of a grammar test suite, and it says the maintainers care about the cases where a real file on disk does not match a clean textbook example.

The version pin between grammar and runtime is the thing to watch

The tree-sitter runtime and the grammar are versioned together, and the metadata makes the required pairing explicit. The npm package declares tree-sitter as a peer dependency at ^0.25.0 and marks it optional in peerDependenciesMeta, so a consumer who already has the runtime installed will not get a second copy. The Cargo dev dependency is tree-sitter at 0.25.8, and the Rust package depends on tree-sitter-language at 0.1, which is the newer interface for exposing a grammar as a language object. On the Python side the optional core extra is tree-sitter~=0.24, which is a slightly wider range than the others. The published releases tell the same story, with v0.25.0 on 2025-09-11 after v0.23.6 in December 2024. The practical advice is to bump the grammar and the runtime in the same change, because the C parser and the tree-sitter library share an ABI that moves between releases.

Where a grammar is the wrong tool

A grammar parses, and it does not execute, so it is the wrong tool if what you need is to run Python or to check semantics. A grammar will accept a program that raises at runtime, and it will accept a program whose types are wrong, because types live in a different layer entirely. The second limit is that tree-sitter grammars are fast but not exacting about the parts of a language that are resolved at runtime rather than at parse time, and for Python that includes things the language spec allows to be ambiguous. The third is that if your tool only ever reads Python that you control, and only needs the standard library's ast module, then a grammar is a large dependency for a small job, and ast is in the standard library. The grammar earns its cost when you need incremental parsing, error tolerance on partially written code, or a query language you can reuse across tools, and it does not when you do not.

Editorial conclusion

tree-sitter-python fits anyone building an editor plugin, a linter, a code navigation tool, or a repository analysis service that needs to read Python reliably and incrementally. It is the wrong tool if you need to execute Python, since a grammar parses and does not run, and it is the wrong tool if you only need the standard library's ast module for a one-off script on a known Python version. Before you adopt it, check the version pairing, because the package declares a peer dependency on tree-sitter at ^0.25.0 and the Cargo dev dependency at 0.25.8, and the bindings for Rust, Node, Python, Swift, and Go all track the same release line.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. tree-sitter/tree-sitter-python on GitHub