chonkiejs: A TypeScript Port of Chonkie for Local Text Chunking

The Gap chonkiejs Fills in a TypeScript RAG Stack

Chunking is the step between raw text and an embedding call, and in a TypeScript codebase it is often the step that forces a decision: call out to a Python process, pull in a library with a native tokenizer dependency, or write a splitter by hand. The README describes the origin directly. The authors were building a TypeScript web app that needed chunking for RAG, and found existing libraries "either too heavy or not flexible enough." chonkiejs is their answer.

The target reader is a TypeScript developer who wants chunking to happen in the same runtime as the rest of the application. The core package is listed with zero dependencies, which matters if you are deploying to an edge runtime or a serverless function where a native binary or a large dependency tree is a problem. The README also notes this is a port rather than a binding, so the API is reimplemented in TypeScript instead of wrapping the Python code.

One Async Factory, One chunk() Call, and a Chunk Array

The architecture is uniform across chunkers. Every class exposes a static async create(options) that returns a configured instance, and every instance exposes chunk(text) returning a Chunk[]. The README's usage example shows the shape: await RecursiveChunker.create({ chunkSize: 512 }), then await chunker.chunk(text), then reading chunk.text and chunk.tokenCount off each result.

The three packages divide the work. @chonkiejs/core holds the local chunkers (Recursive, Token, Sentence, Semantic, Code, Table, Fast) and uses character-based tokenization. @chonkiejs/cloud routes to api.chonkie.ai for Semantic, Neural, Code and other cloud chunkers, and depends on core. @chonkiejs/token adds HuggingFace tokenizer support to the core chunkers and depends on @huggingface/transformers. That split is the main architectural fact to internalise: the zero-dependency core does not ship a real tokenizer, and you pay for one only if you add the token package.

The RecursiveChunker is described as the most general-purpose: it splits through a hierarchy of paragraphs, sentences, punctuation, words, then characters. The SemanticChunker takes a different route. It computes embedding similarity across sliding sentence windows and splits at low-similarity valleys, which is why its create() call requires an embeddings function rather than a model name. You supply the embedding implementation; the chunker only decides where the boundaries fall.

Installing the Core Package and Choosing a Tokenizer

Installation is a single npm command for the local chunkers:

npm install @chonkiejs/core

From there the README's example is three lines of setup and one call:

import { RecursiveChunker } from '@chonkiejs/core'; const chunker = await RecursiveChunker.create({ chunkSize: 512 }); const chunks = await chunker.chunk('Your text here...');

The config keys differ per chunker and are worth reading before you pick one. TokenChunker takes chunkSize (default 512), chunkOverlap (default 0) and tokenizer (default 'character'). RecursiveChunker takes chunkSize, tokenizer, minCharactersPerChunk (default 24) and an optional rules object for a custom split hierarchy. SentenceChunker takes chunkSize (default 2048), chunkOverlap, minSentencesPerChunk, minCharactersPerSentence, a delim array defaulting to ['. ', '! ', '? ', '\n'], and includeDelim set to 'prev', 'next' or 'none'. SemanticChunker adds threshold (default 0.8, lower means more splits) and similarityWindow (default 3).

Two of these need extra setup. SemanticChunker requires an embeddings function or an object with an embed(texts) method, so it cannot run without a model you provide. CodeChunker requires web-tree-sitter and a language grammar; the README says passing a language id requires the tree-sitter-wasms package, and that chunk() is synchronous after create().

The Character Tokenizer Default Changes What chunkSize Means

The default tokenizer value across the documented chunkers is the string 'character'. That is the single most consequential detail in the README, because chunkSize is expressed in tokens. If the tokenizer is counting characters, then chunkSize: 512 means roughly 512 characters, not 512 model tokens. For English prose those two numbers differ by a factor of about four.

The practical consequence: a pipeline tuned with chunkSize: 512 on the character tokenizer will produce chunks that are far smaller than 512 tokens once you swap in a real tokenizer through @chonkiejs/token. If you are moving from the default to HuggingFace tokenizers, expect the effective chunk size to grow and re-check your overlap settings, since chunkOverlap is counted in the same units.

This is not a defect, and the README is explicit that core uses character-based tokenization while the token package adds HuggingFace support. But it is the kind of default that is easy to miss, and it means any comparison between chonkiejs chunk sizes and chunk sizes from a Python library using a real tokenizer is not apples to apples unless you have installed @chonkiejs/token.

Where the Port Falls Short of the Python Original

The README carries a note in its own words: the library is "still under active development and not at feature parity with the original chonkie library yet." That is the honest framing of the main limitation, and it should shape how you evaluate the project.

The release history supports the read. The listed releases are versioned per package (token-v0.0.4, core-v0.0.11, core-v0.0.10), all in the 0.0.x range. There is no 1.0 and no stability commitment in the material. If your chunking behaviour is load-bearing, a 0.0.x version line means you should pin exact versions and read the changelog between bumps rather than relying on semver.

The other gap is tokenization fidelity. Because the core package does not bundle a real tokenizer, the default path does not match what an embedding model will actually see. For retrieval quality work, where chunk boundaries interact with how the model tokenizes text, the character default is a coarse approximation. You can close the gap with @chonkiejs/token, but that pulls in @huggingface/transformers, which is a much larger dependency than the zero-dependency core. The lightweight claim and the accurate-tokenization claim pull in opposite directions, and you have to pick which one you need.

chonkiejs Against LangChain's TypeScript Text Splitters

The obvious alternative in this space is the text splitter modules in LangChain's JavaScript/TypeScript packages, which offer RecursiveCharacterTextSplitter and related classes. The difference in approach is structural rather than a matter of which splits better.

LangChain's splitters are components inside a larger framework. Adopting them usually means adopting the surrounding abstractions for documents, loaders and chains, or at least accepting a dependency on the packages that carry them. chonkiejs is a standalone library with a single static create() call and a chunk() method, and its core package is listed with zero dependencies. If all you want is to turn a string into an array of chunks, chonkiejs gives you less surface area to learn and less to keep current.

The trade-off runs the other way too. LangChain's ecosystem includes retrievers, vector store integrations and evaluation tooling that chonkiejs does not attempt to provide. And chonkiejs's own split against the Python chonkie library is worth stating plainly: the Python version is the reference implementation and, per the README, further along. A team already running Python for embedding or indexing may find the port adds a second implementation of the same idea rather than removing work.

Maintenance, Licence and What to Pin

The repository is MIT licensed, which permits commercial and closed-source use with the usual requirement to carry the licence notice. That is the standard permissive arrangement and it fits the library's role as a dependency rather than a product. This is a description of the licence identifier, not legal advice; read the LICENSE file for the actual terms.

Upgrade cost is shaped by the per-package versioning. Because @chonkiejs/core, @chonkiejs/token and @chonkiejs/cloud release independently, a bump in one does not imply a bump in the others, and the cloud package depends on core. Pin exact versions in package.json and read the release notes for the specific package you are upgrading. The last push date in the repository metadata is 2026-09-10 and the most recent listed release is token-v0.0.4 from 2026-07-07, so the project is still being touched, though the release cadence visible in the material is modest.

The dependency decision is the real maintenance question. Staying on @chonkiejs/core alone keeps the tree empty but keeps you on the character tokenizer. Adding @chonkiejs/token brings in @huggingface/transformers, and that is the dependency you will be tracking and updating over time. Decide which of those two you are committing to before you write the first create() call, because switching later changes your chunk sizes.

Editorial conclusion

Adopt chonkiejs if you are building a TypeScript RAG pipeline and want chunking to run in-process without a Python service or a native binding. Do not adopt it if you need the full chunker set and behaviour of the Python chonkie library, since the README states the port is not at parity. Before committing, verify the tokenizer behaviour you actually need: the default is 'character', and HuggingFace tokenizers live in a separate package, @chonkiejs/token.

Official sources

1. feyninc/chonkiejs on GitHub
2. License: MIT
3. Project website
4. README
5. Releases