json-machine: a lazy JSON parser for PHP that never loads the whole document

The one-line difference between exhausting memory and not

JSON Machine is presented as a drop-in replacement for inefficient iteration of big JSON files or streams, and the README leads with a diff that makes the whole idea legible:

<?php

use \JsonMachine\Items;

// this often causes Allowed Memory Size Exhausted,
// because it loads all the items in the JSON into memory
- $users = json_decode(file_get_contents('500MB-users.json'));

// this has very small memory footprint no matter the file size
// because it loads items into memory one by one
+ $users = Items::fromFile('500MB-users.json');

foreach ($users as $id => $user) {
    // just process $user as usual
    var_dump($user->name);
}

The API surface is deliberately identical. `Items::fromFile` returns something you can `foreach` over, and the items come back as decoded objects. The difference is entirely in what happens under the hood, and it is a generator-based parser rather than a decoder that materialises a tree.

The stated features follow from that: a constant memory footprint for unpredictably large documents, ease of use with no events and callbacks, efficient iteration on any subtree specified by JSON Pointer, and speed from using the native `json_decode` to decode the document item by item.

It also parses not only streams but any iterable that produces JSON chunks, which is what makes it usable against an HTTP response.

The requirements are modest: PHP 7.2 or newer, and no dependencies in production except the optional `ext-json` extension. The README claims the documentation is kept in sync with the code, and the test suite is described as more than 200 tests and 1000 assertions.

Objects and arrays iterate the same way

A JSON object becomes a keyed iteration, and a JSON array becomes a numeric one. Given a document with an `apple` and a `pear` key, the iteration is:

<?php

use \JsonMachine\Items;

$fruits = Items::fromFile('fruits.json');

foreach ($fruits as $name => $data) {
    // 1st iteration: $name === "apple" and $data->color === "red"
    // 2nd iteration: $name === "pear" and $data->color === "yellow"
}

The README states that parsing a JSON array instead of a JSON object follows the same logic, with the key in the foreach being a numeric index of an item.

The items come back as objects by default. If you want arrays instead, the README says to use `new ExtJsonDecoder(true)` as a decoder:

<?php

use JsonMachine\JsonDecoder\ExtJsonDecoder;
use JsonMachine\Items;

$objects = Items::fromFile('path/to.json', ['decoder' => new ExtJsonDecoder(true)]);

That decoder is also the answer to the extension question. The TL;DR section notes that `ext-json` is required if the library is used out of the box but is not required if a custom decoder is used, and the Decoders section documents the available ones. So the native extension is an optimisation and the default, not a hard requirement, and a pure-PHP decoder is the fallback for a build without it.

The performance note in the introduction says the critical code is optimized for speed with JIT in mind, and that it uses native `json_decode` to decode JSON items by default.

JSON Pointer selects a subtree, and the hyphen is a wildcard

The pointer option is the feature that makes the parser useful on real API payloads, because a response usually has a wrapper object around the array you actually want.

Pointing at a subtree is a single option:

<?php

use \JsonMachine\Items;

$fruits = Items::fromFile('fruits.json', ['pointer' => '/results']);
foreach ($fruits as $name => $data) {
    // The same as above, which means:
    // 1st iteration: $name === "apple" and $data->color === "red"
    // 2nd iteration: $name === "pear" and $data->color === "yellow"
}

The README draws attention to what does not happen: the value of `results` is not loaded into memory at once, only one item in `results` at a time, so the memory consumption stays constant at the level you are iterating.

The JSON Pointer specification also allows a hyphen instead of a specific array index. JSON Machine interprets it as a wildcard that matches any array index, and explicitly not any object key. That is what lets you iterate a nested value inside an array of objects without loading the whole item:

<?php

use \JsonMachine\Items;

$fruits = Items::fromFile('fruitsArray.json', ['pointer' => '/results/-/color']);

The wildcard distinction between array index and object key is the sort of detail that matters when you are pointing at a document whose shape varies. The README includes a section explaining what JSON Pointer is, for readers who are not familiar with the notation.

Random access is the thing you give up

The TL;DR section states the limitation plainly: random access like `$users[42]` is not yet possible. The advice is to use the foreach loop and find the item, or to use JSON Pointer.

That is a real constraint on the design rather than a missing feature, because a parser that has constant memory cannot also index into the document it has discarded. Once you have iterated past an item, it is gone unless you kept it.

The same reasoning applies to counting. Counting the items is possible via `iterator_count($users)`, and the README is careful to add that it will still have to iterate the whole thing internally to get the count, and will therefore take about the same time as iterating it and counting in a loop. So the count is available, it is just not free.

The repository has an example file for the memory behaviour, `examples/memLeak.php`, alongside `examples/guzzleHttp.php` and `examples/symfonyHttpClient.php` for the streaming cases. The troubleshooting section in the README is organised around the failure mode people actually hit, with headings for the allowed memory size exhausted error, for the case where that did not help, and for the case where the reader is still out of luck. That troubleshooting structure is a sign the library's users hit memory limits often enough for the author to write a three-step escalation about it.

Streaming an API response instead of a file

Because the parser accepts any iterable that produces JSON chunks, it works against a streaming HTTP response rather than only against a local file. The README has a dedicated section for parsing streaming responses from a JSON API, with separate examples for GuzzleHttp and Symfony HttpClient.

The shape of that integration is the point: instead of asking the HTTP client for the whole body, you hand the response body to the parser and iterate it, which keeps the peak memory of the request at the size of one item rather than the size of the payload.

The options are passed as an array to the `Items` factory, so the pointer and the decoder are configured the same way for a file and for a stream.

There is also a section on tracking the parsing progress, which suggests the library can report how far it has got through the document, useful for a long-running import where you want to show a progress bar or write a checkpoint.

Recursive iteration arrived in version 1.2.0, and the README flags it as new at the top of the document with a link to the recursive section. That is the one feature added in the 1.2 line that the release notes call out explicitly.

The build matrix spans PHP 7.2 to 8.5

The Makefile shows how seriously the compatibility claim is taken, and it does it with containers rather than with a compatibility layer.

The `tests-all` target loops over a list of PHP versions and runs the test suite in each, with the list defined in the Makefile as 7.2.34, 7.3.33, 7.4.33, 8.0.30, 8.1.34, 8.2.30, 8.3.30, 8.4.19 and 8.5.4, each paired with a Composer version. Every invocation goes through `build/docker-run.sh` with an image built by `build/build-image.sh`, so the matrix is genuinely exercised rather than inferred.

The rest of the targets are the standard PHP quality gates. `cs-check` and `cs-fix` run the code style checks through the PHP CS Fixer configuration in `.php-cs-fixer.dist.php`. `phpstan` runs static analysis with the configuration in `phpstan.neon`. `tests` runs on a recent PHP version, `tests-coverage` runs on an older one and produces `clover.xml`, and `performance-tests` is a separate target, which matches the README's emphasis on speed. The `build` target chains `composer-update cs-check phpstan tests-all` and is described as running everything necessary before a commit.

The repository also carries `phpunit.xml.dist` and a `.gitattributes` file, and the `src/` and `test/` directories hold the implementation and its tests.

The library is published on Packagist as `halaxa/json-machine` and the README header carries the monthly downloads badge, the latest stable version badge and a codecov badge.

Where json-machine is the wrong choice

The first wrong case is a workload that needs random access or a second pass. The parser is a generator, so once you have iterated past an item you cannot go back to it without re-parsing. A job that needs the total count before it starts, or that reads the same document twice in different orders, is paying for a full second parse.

The second wrong case is a payload that comfortably fits in memory. If the document is a few megabytes, `json_decode` is simpler, faster and gives you an array you can index and pass around. json-machine is an answer to a memory problem, and adopting it without that problem adds an unfamiliar API for no gain.

The third is a codebase that has to run on PHP below 7.2. The README states the requirement as PHP 7.2 and newer, and the Makefile matrix starts at 7.2.34, so the floor is real.

The fourth is a project that cannot use the native `ext-json` extension and does not want to write a decoder. The default decoder needs it, and while the README says a custom decoder removes the requirement, that is code you own and must maintain.

The fifth is a team that expects the iterator to be cheap to create and cheap to abandon. The parser is lazy, so the work of reading a document happens as you iterate, not when you call `Items::fromFile`. Code that creates the iterator, does something else, and comes back later will find the parsing is interleaved with that other work, and an exception mid-iteration leaves the stream in whatever state the underlying resource is in.

What json-machine does well is narrow and important: it turns an operation that cannot be done into one that can, and it does that with a foreach loop rather than a new programming model.

Editorial conclusion

json-machine suits a PHP team that has to process a JSON file or API response too large to decode in one go, and that wants a familiar foreach loop instead of an event-driven parser. It does not suit a codebase that needs random access to a specific element, since the README states that access like `$users[42]` is not yet possible, and it does not suit a project that wants to hold the parsed document in memory anyway. Before adopting it, check whether your PHP build has `ext-json`, since the default decoder needs it and a custom decoder is required without it, and measure your own payload with the `examples/memLeak.php` script the repository ships, because the memory guarantee applies to the parser and not to the objects your loop accumulates.

Frequently asked questions

Official sources

1. halaxa/json-machine on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases