tree-sitter-markdown splits Markdown into block and inline grammars

The project tells you not to trust it for correctness

The goals section is unusually direct and it should shape your evaluation. Even though this parser has existed for some while and obvious issues are mostly solved, there are still lots of inaccuracies in the output, and these stem from restricting a complex format such as Markdown to the quite restricting tree-sitter parsing rules. As such it is not recommended to use this parser where correctness is important. The main goal for this parser is to provide syntactical information for syntax highlighting in parsers such as neovim and helix. That is a legitimate and narrow purpose, and a project that states its own limits plainly is more useful than one that claims to parse Markdown correctly. The inaccuracies come from the structure of tree-sitter itself, which parses with a context-free-ish grammar and a restricted node set, while Markdown's real structure depends on inline parsing that decides what a line means, so a tree-sitter grammar has to approximate. For highlighting, an approximation that gets the code spans right is enough. For anything that transforms or validates a document, it is not.

Extensions are compile time switches, and the defaults are a decision

The extensions table is the part to read before configuring anything. Extensions can be enabled at compile time through environment variables, and some of them are on by default, which can be disabled with the environment variable NO_DEFAULT_EXTENSIONS. The defaults cover Github flavored markdown with EXTENSION_GFM, task lists with EXTENSION_TASK_LIST, strikethrough with EXTENSION_STRIKETHROUGH, pipe tables with EXTENSION_PIPE_TABLE, YAML metadata with EXTENSION_MINUS_METADATA and TOML metadata with EXTENSION_PLUS_METADATA. The table also records what each one enables, so GitHub flavored markdown pulls in task lists, strikethrough and pipe tables. Two extensions are off by default and have to be asked for. Tags with EXTENSION_TAGS and wiki link with EXTENSION_WIKI_LINK, both following the Obsidian conventions. So the default build handles the GitHub ecosystem and the front matter conventions of static site generators, and leaves the Obsidian-specific syntax out unless you compile it in. Each row also links to its specification, which is a good sign for a project that cares about matching the specs rather than approximating them.

Two grammars and a two pass parse

The standalone usage section explains the one piece of machinery you need to know to use this outside an editor. To use the two grammars, you first parse the document with the block grammar. Then you perform a second parse with the inline grammar using ts_parser_set_included_ranges to specify which parts are inline content. Those parts are marked as inline nodes, and children of those inline nodes should be excluded from those ranges. An example implementation is in lib.rs in the bindings folder. The reason for the split is structural rather than a matter of taste. Inline content, the emphasis markers, the code spans, the links inside a paragraph, determines how the surrounding block is parsed, so a single tree-sitter grammar cannot express the dependency in either direction. Splitting it into a block pass that marks where inline content lives and an inline pass that parses inside those ranges is the standard workaround. The cost is that the consumer has to do two passes and has to be careful about the range boundaries, since including the children of an inline node in the second range would make the inline parser see text it should not.

WASM does not work out of the box

The WASM section is a short but important warning for anyone planning to run this in a browser or in a web based editor. Using this parser with WASM or web-tree-sitter does not work out of the box at the moment, because the parser uses some C functions that are not exported by tree-sitter by default. The fix is to statically link the parser to tree-sitter, and the README points at three issues for detail, one against the tree-sitter repository itself and two against this parser. That is a real constraint for the web tooling market, where web-tree-sitter is the normal route. The node package manifest reflects the same idea in a different place, with a prestart script that runs tree-sitter build --wasm and a start script that launches the tree-sitter playground, so the developer workflow assumes a locally built wasm binary rather than the published shared library. The editor integrations listed for neovim and helix are the intended targets, and both build against a native parser, which is why the wasm limitation has not stopped the project.

Published for three ecosystems with different names

The parser ships as three packages and the naming is not consistent, which is worth knowing before you go looking for it. The npm package is @tree-sitter-grammars/tree-sitter-markdown at version 0.5.3, with the main and types entries pointing at bindings/node, node-addon-api and node-gyp-build as its dependencies, tree-sitter-cli and prebuildify as dev dependencies, and tree-sitter as an optional peer dependency. The Rust crate is called tree-sitter-md, also at 0.5.3, with tree-sitter-language and an optional tree-sitter dependency behind a parser feature, and it includes a benchmark binary behind that same feature. The Python package is tree-sitter-markdown, with a core optional dependency on tree-sitter and Python 3.10 or newer required. The setup.py shows the Python extension being compiled with the C11 standard and hidden visibility, or with the MSVC equivalents, and the wheel tag logic that pins a CPython 3.10 abi3 build on platforms where the GIL is enabled. The repository root also holds CMakeLists.txt, a Makefile, a binding.gyp for the node build, Package.swift for Swift, and a tree-sitter.json.

Shared grammar fragments and the query files

The repository layout shows a common/ directory alongside the two grammar directories, tree-sitter-markdown/ and tree-sitter-markdown-inline/, and the shared part is where the HTML entity table lives. The node package manifest lists common/common.js and common/html_entities.json among the files it publishes, and the Rust manifest includes the same two files plus the two grammar.js files and the query directories, which tells you the shared pieces are needed at build time as well as at runtime. The queries are the part that matters for editors, since a tree-sitter grammar is only half of the story and the other half is the .scm query files that say which nodes to highlight as which token type. The Python build includes the queries as package data, with the .scm files under tree_sitter_markdown.queries, and the egg_info step recursively includes them. The repository also carries a .tsqueryrc.json for query linting and a scripts/ directory, and the CI badge points at a ci.yml workflow.

Editorial conclusion

Adopt tree-sitter-markdown when you are wiring a tree-sitter grammar into an editor such as neovim or helix and want Markdown highlighted with the extensions you enable, and you can live with an approximation of the spec. Do not adopt it where correctness is important, because the project states that directly. Verify first which extensions you need and enable them at compile time, since tags and wiki links are off by default while the GitHub flavoured ones are on, and that you understand the two-pass parsing requirement, since the inline grammar is applied over the ranges marked as inline by the block grammar.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. tree-sitter-grammars/tree-sitter-markdown on GitHub