RAG text chunker
Split a document into overlapping chunks exactly as LangChain's RecursiveCharacterTextSplitter would, with Chinese punctuation understood.
Input
Result
The result will appear here.How a document is cut up decides what a RAG system can retrieve: chunks that stop mid-sentence or mix two topics make poor search results, however good the embedding model is. This tool lets you see the chunks before you index anything. It is a line-by-line port of LangChain's RecursiveCharacterTextSplitter (langchain-ai/langchain, MIT), the splitter most RAG tutorials and pipelines start from, and our tests check that it produces exactly the chunks the Python library does on English, Chinese and mixed text, measured in characters or in tokens.
How it works
- The text is split at the first separator that occurs in it — blank line, line break, then sentence ends, clause marks and spaces — and any piece still too long is split again with the next separator down.
- Chinese sentence and clause punctuation (。！？；，) is on the separator list, and every separator stays at the end of the piece it closes, so a chunk ends with its full stop instead of the next chunk starting with it.
- Pieces are then merged up to the chunk size, and each new chunk starts with as much of the previous one's tail as fits in the overlap, as LangChain does.
- Length is counted in Unicode code points, like Python's len(), or in cl100k_base tokens; the output is JSON with each chunk's index, start offset, length and text.
Where your data goes
Nowhere. This tool runs entirely in your browser: the text you paste is processed by the page and is never transmitted to a server or written to a log.
This tool is free and needs no account. Its results exist only in your open page and are not saved anywhere.
What it costs
This tool is free, with no sign-in and no points.
Common questions
- Will I get the same chunks in my Python pipeline?
- Yes, with the same settings: RecursiveCharacterTextSplitter(chunk_size=…, chunk_overlap=…, separators=["\n\n", "\n", "。", "！", "？", ". ", "! ", "? ", "；", "; ", "，", ", ", " ", ""], keep_separator="end"). For token mode add length_function counting with tiktoken's cl100k_base. LangChain's own default separators ignore Chinese punctuation, which is why Chinese text splits badly with them.
- What chunk size and overlap should I use?
- A common starting point is 500–1,000 characters, or 200–400 tokens, with an overlap of 10–20%. Smaller chunks give more precise matches but less context per hit; larger ones do the reverse. Try your real documents here and read the chunks: each should make sense on its own.
- Why is a chunk sometimes shorter than the overlap would suggest?
- The overlap is a maximum, not a guarantee. The splitter only carries whole pieces over, and it drops pieces from the front until the next one fits within the chunk size, so a long sentence may leave no room for overlap at all. LangChain behaves the same way.
- Does it support Markdown headers or semantic chunking?
- No. It does the recursive character split, which is what most pipelines use by default. Header-aware and embedding-based semantic splitters need different logic, and in the semantic case an embedding model; blank lines and headings already act as the strongest boundaries here.
The open-source behind it
This tool is a self-contained implementation. langchain-ai/langchain (MIT) does the same job as a library — if you need this behaviour inside your own program, start there rather than calling a web page.
langchain-ai/langchainAlso known as
- text chunker
- rag chunking
- recursivecharactertextsplitter
- langchain text splitter online
- chunk size overlap
- document chunking