tolerant-php-parser: a PHP parser built to survive the code you are still typing

A parser that treats broken code as the normal case

The design premise is stated in the first of the design goals and it is the whole reason this parser exists. In IDE scenarios, code is by definition incomplete, so in the case that invalid code is entered the parser should still be able to recover and produce a valid and complete tree, as well as relevant diagnostics. That is a different contract from the one a compiler-shaped parser offers, where invalid input is an error and no tree is produced. The practical difference shows up in the way the library is used, and the README example makes it explicit with a comment saying that the parser handles errors gracefully, so it can be used in IDE usage scenarios where code is often incomplete. Diagnostics are then retrieved separately rather than thrown, through a static call that gets diagnostics from the AST node. The example is short enough to read as the whole API:

<?php
// Autoload required classes
require __DIR__ . "/vendor/autoload.php";

use Microsoft\PhpParser\{DiagnosticsProvider, Node, Parser, PositionUtilities};

// Instantiate new parser instance
$parser = new Parser();

// Return and print an AST from string contents
$astNode = $parser->parseSourceFile('<?php /* comment */ echo "hi!"');
var_dump($astNode);

Note what the parse call is given. The source passed in is a complete file, opened with the PHP tag and containing a comment and an echo, and the parser is asked for a source file rather than a fragment, which is the shape an editor would use when it parses a whole file to drive features. The repository states plainly that there is still a ton of work to be done, and that at this point the repository mostly serves as an experiment and the start of a conversation, so a reader should approach it as a reference implementation with a set of ideas behind it rather than as a finished library.

The tree keeps comments and whitespace so it can be printed back

The second property is representativeness, and it is what turns a parse tree into something a formatter can be built on. The AST is described as fully representative and round-trippable back to the text it was parsed from, with all whitespace and comment trivia included in the parse tree. The reason this matters is that a tree that drops trivia cannot be used to reformat code, because the formatting information has been thrown away. The README example shows the mechanism, a call that gets the leading comment and whitespace text of a node, which is the kind of access a formatter needs and that a conventional AST does not offer. Two supporting design decisions make the tree cheap enough to work with. Nodes link back to their parents, so it is easy to navigate the tree, and the tree can be traversed through parent and child nodes. The goal stated for that is that each language server operation should be under 50 ms, on the back of a target of under 100 ms UI response time, leaving room for everything else going on in parallel. That budget is what forces the rest of the design, including the memory-conscious representation of positions.

Positions are a single integer, not a line and character pair

Memory consumption is treated as a first-class design concern, and the positions model is the clearest example. The example code notes that to cut down on memory consumption, positions are represented as a single integer index into the document, but their line and character positions are easily retrieved. The retrieval is a separate static utility, and the call in the example passes the position and the file contents, which is why the file contents have to be reachable from a node. The design goal behind this is that the parser should be fast and lightweight, able to parse several MB of source code per second, to leave room for other features, and the list under it names memory-efficient data structures and allowing for incremental parsing in the future. The incremental parsing item is telling, because incremental parsing is the technique a language server needs to avoid reparsing a whole file on every keystroke, and it is listed as a future intention rather than a current feature. So the position representation is a piece of preparation, and a consumer building on this today should check how much of that preparation is actually usable.

The phased plan, and where the work actually is

The current status section describes an incremental approach intended to keep correctness high at every step, and the checklist is unusually concrete. Phase 1, complete, was a lexer that does not support PHP grammar but supports EOF and unknown tokens, with tests for all invariants. Phase 2, complete, added the PHP lexical grammar with lots of tests. Phase 3, complete, was a parser without PHP grammar that produces a tree of error nodes, again with invariant tests. Phase 4, complete, added the PHP syntactic grammar with lots of tests. Phase 5, in progress, is real-world validation and optimisation, split into correctness work, meaning validating that no errors are produced on sample codebases, benchmarking against other parsers and investigating any instance of disagreement, plus fuzz-testing, and performance work, meaning profiling and benchmarking against large PHP applications. Phase 6, not started, is finalising the API. That last point is the one to weigh before building anything, and the README says so directly, that the API is not yet finalised, and asks people to file issues about what functionality they want exposed. The additional notes name the specific gaps: yield expressions and template strings are not yet supported, and there are other miscellaneous bugs, but because the parser is error-tolerant these are handled gracefully and the resulting tree is otherwise complete.

A validation suite and a syntax visualizer as the feedback loop

There is a way to see how the parser handles real code rather than curated examples, and the documentation points at it. The validation test suite can be run to get an overall sense of where the parser stands, and the current validation test results are published. Contributing.md covers how to run the tests, and there is a performance test section that the README suggests running on your own machine to see the approach for yourself. The repository layout backs this up with a validation directory, an experiments directory, a tools directory, a syntax-visualizer directory, a tests directory and a ci directory, alongside a phpstan.neon for static analysis and a phpunit.xml for the test runner. The syntax visualizer in particular is worth noting, because a parser that must round-trip arbitrary source is exactly the kind of thing where a visual view of the tree makes a bug obvious in seconds. The invariants mentioned in the design goals are also worth understanding as a concept rather than a detail. The goal is that the parser should produce provably valid parse trees, and that is achieved by defining and continuously testing a set of invariants about the tree, which is a stronger claim than having a test suite that happens to pass.

Written in PHP, and the trade-off that comes with it

The last design goal is about the implementation language and it is a deliberate choice rather than an accident. The parser is written in PHP to make it as easy as possible for the PHP community to consume and contribute. The benefit is that anyone who writes PHP can read the source, hack on it, and contribute without learning a new language, and the consuming side is the same argument from the other end, a PHP tool that a PHP project can vendor without a second runtime. The cost is inherent to a parser written in an interpreted language, and the project addresses it head on with the fast and lightweight goal and the several MB per second target. Two other goals in the same list are worth quoting because they are unusual priorities for a parser. Simple and maintainable over time, with the reasoning that parsers have a tendency to get really confusing really fast, so readability and debug-ability is high priority. And a friendly and descriptive API to make it easy for others to build on. The specification support is also stated: it adheres to the PHP language spec, with the php-langspec directory present in the repository as a submodule, and it supports both the PHP5 and PHP7 grammars. The v0.1 branch note explains the current data structures change to support syntax added after the initial 0.0.x release line.

Editorial conclusion

tolerant-php-parser is a good fit for a team building a language server, a formatter or a refactoring tool for PHP, since the error tolerance and the round-trippable tree with its trivia are the two properties that such tools need and that a conventional parser does not provide. It is the wrong tool for a project that needs a validated AST of running code, since the tree can contain error nodes by design, and it is the wrong tool for anything that needs a stable API, because the API is explicitly not finalised. Before adopting it, read the phase list, because phase 5 for real-world validation and optimisation is still in progress and phase 6 for finalising the API has not started, and expect to file issues describing the functionality you need exposed. The licence is MIT, the repository publishes no GitHub releases, and the last push was on 2026-08-31.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. microsoft/tolerant-php-parser on GitHub
4. README