stream-json: process JSON documents larger than memory without loading them

Tokens, not a tree

The pitch in the first paragraph of the README is the whole idea. Point stream-json at a document far larger than available memory and it streams straight through. You pick out only the parts you care about and handle them one at a time, instead of loading the whole thing with `JSON.parse`. Individual keys, strings and numbers can be streamed piece by piece, and a SAX-inspired event API is included.

That is a different contract from `JSON.parse`. A parser that returns a value hands you everything at once and lets the garbage collector deal with the peak. A token stream lets you decide, stage by stage, what is worth materialising, and the bytes you skip are never assembled into memory. The README's word for the filters is surgical, and the claim is concrete: `pick`, `ignore`, `replace` and `filter` keep the subobjects you want out of a massive document and drop the rest.

The package is published on npm as `stream-json`, version 3.7.0, with the description making the scope explicit: a micro-library of components for processing JSON files and streams with a minimal memory footprint, with a SAX-inspired token API, on Node.js or Web Streams. The licence is BSD-3-Clause.

The design also has a clear boundary, stated in a section of its own. stream-json is built for data you own or trust, such as database dumps, exports, logs and files produced by your own systems. It is not designed for hostile input, and the README says not to feed it JSON or JSONC from the open internet or from untrusted users, because untrusted JSON needs validation of its own before it reaches a pipeline. For a library whose job is to turn bytes into structure, that is the right thing to say.

Each component is a pipeline stage, and stream-chain is the wiring

The components fall into three roles. The parser turns text into a token stream. The filters trim and reshape that stream on the fly. The streamers assemble the surviving tokens back into JavaScript objects.

They compose with each other and with your own code through stream-chain, the zero-dependency library this one is built on, which handles the streaming and the backpressure. The README's word for the result is composable: every component is an ordinary pipeline stage, and you can mix them with each other, with plain functions and generators, and with any Node or Web stream.

That dependency is a real architectural choice rather than an implementation detail. Most JSON libraries would be a single package you call; this one is a set of stages that a separate library joins together, which is why `stream-chain` is also the home of the streaming JSONL support. The companion project list names stream-csv-as-json as well, which streams large CSV files into a stream-json compatible token format, rows as arrays of strings, or as objects when a header row is present.

TypeScript typings are bundled, the package is ESM only, and the README says it runs on actively maintained Node, plus Bun and Deno, with a wiki page for the supported runtimes.

The example in the README shows the shape of a real pipeline, and the comment in it is the point of the whole library:

import {parser} from 'stream-json';
import {pick} from 'stream-json/filters/pick.js';
import {streamArray} from 'stream-json/streamers/stream-array.js';
import chain from 'stream-chain';
import fs from 'node:fs';

// data.json: { "meta": {...}, "data": [ ...millions of records... ] }
const pipeline = chain([
  fs.createReadStream('data.json'), // a file far bigger than RAM is fine
  parser(),
  pick({filter: 'data'}), // descend into "data", ignore everything else
  streamArray() // emit one array element at a time
]);

A file stream in, a parser, a filter that descends into one key, and a streamer that emits one array element at a time. The bytes for `meta` are skipped and never become an object.

One match streams an array, several matches need streamValues

There is a subtlety in the API that the README explains carefully, and it is the kind of detail that costs an afternoon if you miss it.

When `pick` selects a single array, `streamArray` streams its elements directly. When `pick` matches several subobjects, the output is a sequence of separate sub-trees, structurally the same token stream that a JSON Streaming source produces, and `streamValues` is what assembles each match into its own object.

import {parser} from 'stream-json';
import {pick} from 'stream-json/filters/pick.js';
import {streamValues} from 'stream-json/streamers/stream-values.js';
import chain from 'stream-chain';
import fs from 'node:fs';

// depts.json: {"departments": [
//   {"name": "dev", "head": {"name": "Alice", "id": 1}, "staff": 20},
//   {"name": "ops", "head": {"name": "Bob", "id": 2}, "staff": 10}
// ]}
const pipeline = chain([
  fs.createReadStream('depts.json'),
  parser(),
  pick({filter: /^departments\.\d+\.head\b/}), // every department's "head" subobject
  streamValues() // assemble each picked sub-tree
]);

The filter is a regular expression matched against the path, so `^departments\.\d+\.head\b` selects the `head` subobject of every department without touching anything else. The result is that each match arrives as a complete object on its own, and the pipeline handler receives `{value}` for each one.

The distinction between `streamArray` and `streamValues` is therefore not cosmetic. `streamArray` expects one array and walks it, `streamValues` expects a sequence of independent sub-trees and assembles each. Choosing the wrong one produces a pipeline that either emits nothing or emits the wrong shape, and the README is unambiguous about which to use in which case.

There are also file helpers for the ends of the pipeline: a Node-only `parseFile()` to read straight from a file, and `stringerToFile()` to write a stream back to disk.

Three builds and three parsers, chosen by the export map

The `package.json` export map shows how the package is split for different environments. The default entry point is `./src/index.js`, there is a `./web` entry point pointing at `./src/web/index.js`, and a wildcard export for `./*` so that the individual components can be imported by path, which is how the example imports `stream-json/filters/pick.js` and `stream-json/streamers/stream-array.js` directly rather than through a barrel file.

The wildcard export has a consequence worth understanding. It means the internal file layout is effectively part of the public API, and it lets you import only the stage you need instead of pulling the whole package into the module graph. For a library whose selling point is a minimal footprint, that is deliberate.

The README lists what is in the box. There are three parsers: JSON, JSONL, which is described as the fast path for line-delimited data dumps, and JSONC, which is JSON with comments. There are helpers for streamed objects, meaning you can pick out the parts you want, assemble objects from tokens, output JSON back, and read or write files directly. And there are three runtime targets: it works out of the box on Node with Node streams, there is a browser-safe build using Web Streams, and there is a dependency-free core build with no streams at all, for when you need to use the components without the stream layer.

The dependency-free core is the piece that distinguishes this from a Node-only tool. The same parsing and filtering logic, with no stream plumbing, is useful in a worker, in a short-lived function, or in an environment where streams are the wrong abstraction.

The repository also carries an `ARCHITECTURE.md`, a `wiki/` directory that the README points to for details, a `bench/` directory, a `dev-docs/` directory, and `llms.txt` and `llms-full.txt` files for language model tooling.

The changelog is where the engineering shows

The release history in the README is short, and it is more informative than a typical changelog because each entry names the change and the contributor.

Version 3.7.0 added a `maxDepth` option on `FlexAssembler` rules, credited to zx, together with significant performance improvements for the string and RegExp filters. Version 3.6.0 carries two security fixes: a `__proto__` key no longer replaces an assembled object's prototype, and JSONC comments no longer rescan. The same release changed how JSONC comments are streamed, as `startComment`, `commentChunk` and `endComment` plus a `commentValue`, with the old `comment` token renamed, made `replace` accept any plain value as its replacement, and made filters replay keys that had been packed again. Version 3.5.0 added a `maxDepth` option to path filters.

Two of those deserve attention. The `__proto__` fix is a prototype pollution fix, and it is the second security fix in the same release, since the JSONC rescan issue is the other. For a library that assembles objects from a token stream, that pair is exactly the class of bug you would expect, and the fact that both were fixed in one release suggests the maintainer was actively looking at that surface.

The `maxDepth` option appears twice, once for the assembler rules and once for the path filters, and it is a safety valve rather than a convenience. A depth limit bounds how deeply a filter or an assembler will recurse into a structure, which matters for a document you do not fully control.

The test setup matches the runtime claims. The scripts run the suite under tape6 with several harnesses, and there are separate scripts for Bun, for Deno, and for sequential and process-based runs, plus a TypeScript test variant that runs the same tests from `.ts` files. The `ts-check` and `js-check` scripts run `tsc` in two configurations, one for the typings and one for a separate checking project.

The repository has no GitHub releases, so the npm version is the release record. The last push was on 2026-10-02.

Where stream-json is the wrong tool

The first wrong case is JSON you do not control. The README is unambiguous that the library is not designed for hostile input and that untrusted JSON needs its own validation before it reaches a pipeline. The `__proto__` fix in 3.6.0 is a good reason for that stance, and it is a reminder that a streaming parser that assembles objects is still an object construction path.

The second is a document that fits comfortably in memory. If your JSON is a few megabytes, `JSON.parse` is faster, simpler and synchronous, and a pipeline of five stages to read a file is a lot of machinery for the job. The library is for the case where the alternative does not work.

The third is a CommonJS codebase. The package is ESM only, with the type module marker set and no CommonJS entry in the export map. A project that cannot move to ESM has to stay on an older major version or a different tool.

The fourth is a job that needs the whole document as a value. This library's core operation is to not have the whole document. If your algorithm genuinely needs a complete object graph, a streaming parser is the wrong shape, and the constant memory property works against you rather than for you.

The fifth is a performance decision made on someone else's numbers. The README says the parser and assemblers are measured and tuned along the hot paths, and then says the real numbers depend on your data and hardware, and points you at a benchmark page to run on your own. That is the correct position for a library author, and it also means the performance case has to be made in your own environment.

Editorial conclusion

Adopt stream-json if you routinely process exports, dumps or logs that do not fit in memory, if you need to select a few paths out of a large document, or if you want to stream JSON in a browser or in Bun or Deno. Do not adopt it for JSON that arrives from untrusted sources, because the README rules that out explicitly, and do not adopt it if you need a CommonJS build, since the package is ESM only. Before you start, pick the runtime build you need from the export map, the default entry point for Node, the `./web` entry point for Web Streams, or the dependency-free core, and write your own validation stage in front of the parser if the data comes from outside your trust boundary.

Frequently asked questions

Official sources

1. Issues
2. License: BSD-3-Clause
3. README
4. uhop/stream-json on GitHub