es-module-lexer: pulling import and export metadata out of JavaScript fast

A lexer, not a parser, and the difference is the whole point

es-module-lexer is described in the README as a JS/TS module syntax lexer used in es-module-shims, and its job is stated precisely: it outputs the list and locations of exports and import specifiers, including dynamic import and import meta expressions.

That is a much narrower job than parsing. A parser builds a syntax tree for the whole file, which means it has to understand every construct in the language, and the cost of that grows with the complexity of the code rather than with the number of module statements in it. A lexer that only cares about module syntax can skip most of the file. The README's framing is that it comprehensively handles the JS language grammar while remaining small and fast, which is a claim about coverage of the grammar's edge cases, not about building a tree of them.

The performance figure in the README makes the difference concrete. Angular 1, at 720KiB, is fully parsed in 1ms, in comparison to the fastest JS parser, Acorn, which takes over 100ms. And the headline rate is about 5ms per MB of JS cold and about 3ms per MB warm.

The package table puts numbers against each of the four entry points. The full build, the es-module-lexer export, is 14.8KiB Brotli-compressed with 4.7ms/MB cold and 3.2ms/MB warm. The js build of the same is 13.1KiB at 14.9ms/MB cold. The minimal build is 7.3KiB at 4.5ms/MB cold and 3.1ms/MB warm, and the minimal js build is 6.5KiB at 10.8ms/MB cold. Footprint is the Brotli-compressed size, and the parse times are per MB of source, measured on the 3.1 MB test/samples set on a standard desktop machine running Node.js 26, with warm averages over 25 parses, via chomp bench.

The interesting thing about those numbers is the shape of the table rather than the values. The WebAssembly builds are three times faster cold than the asm.js builds, and the asm.js builds are smaller by a couple of kilobytes. That is the trade the package is making explicit: you can have the small size of a pure JavaScript build, or the speed of a WebAssembly one, and you can have both only if your runtime supports the right features.

Four entry points, and which one you want depends on your runtime

The package exports in package.json show four entry points, and choosing between them is the first decision an adopter makes.

The main es-module-lexer export is the full build with WebAssembly, JavaScript and TypeScript support. The ./js export is the same full build compiled to asm.js, and it is the fallback for environments where WebAssembly compilation is blocked. The ./minimal export is a smaller build with a v2-like API, in WebAssembly and JavaScript but JavaScript only, meaning no TypeScript lexing. The ./minimal/js export is the asm.js version of that.

The TypeScript point deserves a note. The full build lexes both JavaScript and erasable type-only TypeScript imports and exports, and the README has a dedicated section for it. The minimal build is JavaScript only, so a TypeScript project that needs type-only import handling has to take the larger build.

The environment requirements follow directly from this. The full build requires Node.js 18 or newer and engines with WebAssembly SIMD support, which the README pins as Chrome 91+, Firefox 89+ and Safari 16.4+. The minimal build carries no SIMD requirement and runs in all browsers with baseline ES module support, Chrome 63+, Firefox 67+ and Safari 11.1+, and the asm.js builds cover the environments without WebAssembly at all.

The Content Security Policy story is handled explicitly, which is a sign that the author has run into real deployments. The WebAssembly builds generate no code from strings, so they work with JavaScript eval disabled, including Node.js with --disallow-code-generation-from-strings, as long as the environment permits WebAssembly compilation. For environments or CSP policies that disable WebAssembly compilation, which the README names as a script-src without wasm-unsafe-eval policy, the answer is the js and minimal/js builds:

import { parse } from 'es-module-lexer/js';

So the choice matrix is: fastest if you can compile WebAssembly with SIMD, smallest if you cannot.

The full build's record shape, and the fields worth knowing

The full build returns discriminated unions rather than flat records, and the difference matters if you are porting code.

The README's example is short enough to take in at a glance:

import { init, parse } from 'es-module-lexer';

await init();

const source = `import { a } from './dep.js';\nexport var p = 5;`;
const [imports, exports] = parse(source);

imports[0].specifier; // "./dep.js"
source.slice(imports[0].importStart, imports[0].importEnd); // "import { a } from './dep.js'"
exports[0].name; // "p"

So the full build exposes a specifier property, importStart and importEnd for the statement range, and a name for exports. The minimal build uses the terse v2-style shapes instead, with n for the specifier, ss and se for the statement start and end, s and e for the value range, ls and le for the exported value range, a for the attribute start with -1 meaning no import attributes, and t as a numeric import type.

The README also documents the dynamic import and import.meta conventions in the minimal build, which is where the interesting edge cases live. A dynamic import has d as the start of the expression argument, with -1 for static imports and -2 for import.meta, so a single numeric field tells you which kind of record you are holding. The specifier of a dynamic import is only returned for string literal dynamic imports.

The full build adds more than the minimal one. The minimal build keeps the v2 flat shape with no export classification, no origins, no statement starts and no export * records, which the README states directly. So if you need to know whether an export is a re-export or which file an import came from, that is a full build feature and the minimal build will not give it to you.

The upgrading section lists the specific differences from v2 for the minimal build. init is a function returning a promise rather than a promise itself, so await init becomes await init(), and calls are idempotent and share one compilation. parse returns only imports and exports, with facade and hasModuleSyntax dropped. at is dropped from import records, and attributes are read via source.slice(a, se - 1). ImportType is a type-only union of numeric literals with no runtime export, so StaticImportType is 1 and DynamicImportType is 2. And interpolated template specifiers are not globbed in the minimal build, so n is undefined for them, while escape sequences in specifiers are decoded into n just as in the full build, including the parse error on invalid escape sequences.

Using it, and the init() ordering trap

Installation is a single npm command, and the usage pattern is init then parse.

npm install es-module-lexer

The type definitions live in src/lexer.ts for the full build and in types/lexer.d.ts and types/lexer.minimal.d.ts for the published interfaces, and the package.json points the types field at types/lexer.d.ts.

The init call has a subtlety that the README spells out, and it is the kind of detail that produces confusing bugs if you miss it. While awaiting init() is always recommended since browser main threads restrict synchronous WebAssembly compilation, in Node.js and other environments it may be optional, since parse will rely on synchronous compilation otherwise. And calling parse before a pending init() has resolved will also fall back to synchronous compilation, so avoid mixing the two.

That last sentence is the practical warning. If you kick off init() without awaiting it and then call parse immediately, you get the synchronous path, which in a browser main thread is exactly the case the README says the await exists to avoid. The advice is not to start init() and race it.

The supported modern syntax is also worth listing, because a lexer that missed a syntax feature would silently produce wrong import lists. The README says it supports import attributes, deferred evaluation and source phase imports, plus the dynamic import and import meta expressions. The minimal build example shows an import with an import attribute in the source, import json from './json.json' with { type: 'json' }, and reading the attribute with source.slice(imports[1].a, imports[1].se), which returns the attribute text.

The package metadata also shows the packaging details. The main field is dist/lexer.cjs and the module field is dist/lexer.js, the type is module, and sideEffects is false so a bundler can drop it if it is unused. The published files are dist and types. The build and test scripts both install chomp globally and run chomp build or chomp test, and the build is described in the README as being built with Chomp, with a chompfile.toml at the repository root.

Where a module lexer stops, and what to use instead

The limitations follow directly from the scope, and they are not gaps in the implementation so much as consequences of the decision not to build a tree.

First, it lexes module syntax only. There is no AST, so there is no way to ask what expressions exist in a file, what functions are declared, or what a particular identifier refers to. If your problem is dependency analysis of a module graph, this is exactly right. If your problem is understanding code, it is the wrong tool and always was.

Second, the full build's advantage over the minimal one comes with a cost in the TypeScript case. Type-only import and export handling is in the full build only, and the minimal build is JavaScript only, so a TypeScript codebase that wants type-only imports lexed has to take the 14.8KiB build rather than the 7.3KiB one, and the js variant of the full build costs 14.9ms/MB cold against 4.5ms/MB for the minimal WebAssembly build.

Third, the WebAssembly dependency is a deployment constraint. On a modern runtime the full build is the obvious choice. In an environment with a strict CSP, in a browser extension, or in a build tool that has to run in a locked-down sandbox, the asm.js fallback is slower by roughly a factor of three and you are giving up some of the reason to use this package rather than Acorn. The README is upfront that the WebAssembly builds are the fast ones, and it tells you exactly how to fall back.

Fourth, the record shapes are not stable across entry points. Code written against the full build's specifier and importStart properties does not work against the minimal build's n and ss, and the v2-to-v3 migration for the minimal build changed init from a promise to a function, dropped facade and hasModuleSyntax, and dropped at. A library that exposes both builds has to support both shapes, which is a maintenance burden for the consumer rather than for the author.

Fifth, and most simply, this is a lexer for a specific grammar. It is not a general JavaScript analysis tool, and it will not answer questions about code that happens to be in a module file.

es-module-lexer against a full parser

The comparison is with a general-purpose JavaScript parser such as Acorn, and the difference in approach is the difference between reading the module statements and reading the file.

Acorn builds a full syntax tree. That tree is what you need if you want to analyse code, run a linter, transform source, or answer questions about what a program does. It is also why the README's comparison is stark: Acorn takes over 100ms on the same 720KiB Angular 1 file that this lexer handles in 1ms. The ratio is not a tuning difference. It is the difference between allocating and linking a tree of every node in the file and scanning for the small subset of constructs you care about.

The consequence is that the two are not really competitors. A bundler that needs to rewrite import specifiers needs a tree, or at least needs to know every import statement and where its specifier is, and the second part is exactly what this lexer provides. A loader that wants to know what a module depends on, so it can fetch those dependencies in the right order, needs the import list and the specifier strings, and again the lexer is enough. A tool that wants to know whether a file contains a call to a given function is asking a question the lexer cannot answer at all.

There is a third option, and it is worth naming because it is what the author built. es-module-shims is the project that uses this lexer, and the README names the minimal build as the one size-sensitive embedders such as es-module-shims reach for. That is a runtime use case a general parser does not fit, and it is also a case where the footprint matters.

So the decision comes down to the question you are asking. If the question is what does this file import and export, this lexer answers it in milliseconds and a few kilobytes, and nothing else in the ecosystem does it that cheaply. If the question is what does this code do, you want a parser, and no amount of speed in a lexer will get you there.

Editorial conclusion

Adopt es-module-lexer if you need to know what a JavaScript module imports and exports without a full parse, especially in a browser or a loader that must run under a strict Content Security Policy. Do not adopt it if you need a complete syntax tree, type information, or accurate positions for arbitrary expressions, since it lexes module syntax only. Verify first that your runtime supports WebAssembly SIMD if you intend to use the full build, that the record shape your code expects matches the entry point you import, since the minimal and full builds use different field names, and that the version you pin is one whose breaking changes you have read.

Frequently asked questions

Official sources

1. guybedford/es-module-lexer on GitHub
2. Issues
3. License: MIT
4. README
5. Releases