open-parse: a visual chunker for PDFs that keeps layout and tables in the loop
Improved file parsing for LLM’s
At a glance
- What is it?
- open-parse parses PDFs into pydantic nodes, then groups them into chunks using layout and optional embeddings. It is a good fit for RAG pipelines over structured documents, and a poor fit if you need OCR without installing Tesseract or a fully deterministic, offline-only parser.
- Who is it for?
- Adopt open-parse if your RAG pipeline ingests structured PDFs and you want chunk boundaries that follow headings, sections and tables rather than token counts. Do not adopt it if you cannot install Tesseract for scanned documents, or if you need a guaranteed offline parser with no embedding API calls.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 121 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem open-parse targets: chunking that respects document structure
Most chunking code in a RAG pipeline starts by converting a file to raw text and slicing it by token count. The README frames the cost of that approach directly: you lose the ability to overlay a chunk back onto the original PDF, you discard headings, sections and bullets, and you get no handling for tables, images or markdown. open-parse is aimed at teams whose source documents are not flat prose. Manuals, reports and filings have visual structure, and a chunk that cuts across a table or splits a heading from its body is a chunk that retrieves badly. The library's stated goal is to visually discern document layouts and chunk them the way a human would. The audience is narrow but real: Python engineers building retrieval over PDFs who have already hit the ceiling of naive text splitting and want a library rather than a hosted document API.
How open-parse works: nodes first, grouping second
The pipeline has two distinct stages. The first is parsing. A DocumentParser takes a file path and returns a parsed document whose content is exposed as nodes. The basic example in the README is short: construct openparse.DocumentParser(), call parser.parse(basic_doc_path), then iterate over parsed_basic_doc.nodes. Each node is a unit of content with its layout information retained, which is what makes overlay back onto the source PDF possible in principle. The second stage is grouping. The README describes chunking as grouping similar semantic nodes together, and the optional SemanticIngestionPipeline does this by embedding each node's text and clustering nodes by similarity. That separation matters: parsing and grouping are configured independently, so you can run the parser alone and do your own grouping, or hand the parser a processing_pipeline and let it cluster for you. Results are pydantic models, so parsed_content.dict() and parsed_content.json() are the serialization paths. Tables are handled as a separate concern with their own argument block, and the README points at three backends: PyMuPDF's table detection, Microsoft's Table Transformer, and unitable. The table examples in the README are described as parsed with unitable, and the library calls unitable a transformers-based approach with state-of-the-art performance. That claim comes from the project's own documentation, not from an independent measurement.
Getting it running: pip, Tesseract and the ml extra
The core install is a single command: pip install openparse. Python 3.8 or newer is required. PDF handling is built on pdfminer.six, which the README describes as fully open source. OCR is where installation gets fiddly. PyMuPDF contains the OCR logic, but Tesseract's language data has to be present, and the language support folder location must be communicated either through the TESSDATA_PREFIX environment variable or as a parameter to the relevant functions. The README lists typical locations: C:/Program Files/Tesseract-OCR/tessdata on Windows, /usr/share/tesseract-ocr/5/tessdata on Unix, and on macOS via Homebrew either /opt/homebrew/share/tessdata or a version-specific path under /opt/homebrew/Cellar/tesseract/<version>/share/tessdata/. On macOS the README suggests export TESSDATA_PREFIX=$(brew --prefix tesseract)/share/tessdata. On Windows it uses setx, and adds a warning worth repeating: this must happen outside Python before starting your script, because manipulating os.environ inside a running interpreter will not work. Table extraction through deep learning models is a separate extra: pip install "openparse[ml]", followed by openparse-download to fetch model weights. The parser is then configured with a table_args dictionary containing a parsing_algorithm key, for which the README shows the value "unitable".
Semantic chunking depends on an embedding provider
The SemanticIngestionPipeline is the part of open-parse that most changes your deployment shape. The README's example passes openai_api_key, a model name (text-embedding-3-large in the sample), and two bounds, min_tokens=64 and max_tokens=1024. Those bounds are the practical control over chunk size: nodes below the minimum are presumably merged upward and nodes above the maximum are split, though the README does not spell out the merge and split rules in the excerpt available here. The important consequence is architectural. If you enable this pipeline, parsing is no longer a local operation. Every node's text goes to an embedding endpoint, which means document content leaves your infrastructure, cost scales with node count rather than page count, and parse latency now includes network round trips. The README's criticism of commercial document services includes the line that they require sharing your data with a vendor. That criticism is fair for the hosted products, but teams enabling the semantic pipeline should notice they are making a similar trade with their embedding provider. If that is unacceptable, the parser still runs without a processing_pipeline, and you can cluster nodes yourself.
Where open-parse is the wrong tool
Three cases stand out. The first is scanned documents in an environment where you cannot install Tesseract or set TESSDATA_PREFIX before the process starts. The README is explicit that OCR requires Tesseract's language support data, and the Windows note rules out setting the variable from inside Python. Container images and serverless runtimes that only let you set environment variables at request time will need the variable baked into the image instead. The second case is deterministic, offline-only pipelines. The core parser is local, but the semantic pipeline calls an embedding API, and the ml table path downloads model weights with openparse-download. Both introduce external dependencies that a pure pdfminer.six or PyMuPDF text extraction path does not have. The third case is documents that are already clean, structured text. If your inputs are markdown or well-formed HTML, the visual layout analysis is overhead. The README itself acknowledges that layout parsers exist and are good at identifying elements; the gap open-parse claims to fill is grouping related content, not detection. If you only need detection, a layout parser alone may be simpler.
Compared with layout-parser and with hosted document APIs
The README draws two contrasts. Against layout parsers such as layout-parser, the argument is one of scope: those tools identify text blocks, images and tables, but the README states they are not built to group related content effectively, and that you would need to add another model to extract markdown from images, parse tables and group nodes. The README also reports that the authors found performance sub-optimal on many documents and computationally heavy, which is their assessment rather than a published benchmark. The practical difference is that open-parse ships the grouping stage as part of the library, with a pydantic node model as the interface between stages. Against commercial services such as Google Document AI, AWS Textract and Reducto, the README cites a typical price of roughly $10 per 1,000 pages and the requirement to share data with a vendor. The difference in approach is not accuracy claims; it is where the work runs. open-parse is a library you install and run on your own machines, which means you own the compute, the Tesseract installation and the model weights, and you avoid per-page pricing. For low volumes, the hosted services may be cheaper once you count the engineering time spent on Tesseract paths and ml extras.
Maintenance, licensing and what the repository tells you
open-parse is MIT licensed, and the repository is not archived. The most recent release listed is v0.7.0 from November 2024, preceded by v0.6.1 and v0.6.0 in the same autumn. The last push timestamp shown is May 2026, which is later than the newest release, so the main branch has moved since the last tagged version. That gap between tags and commits is worth checking before you pin a version: if you install from PyPI you get v0.7.0 or later, and if you install from git you get unreleased code. Licence implications beyond the MIT terms come from the dependencies, not from open-parse itself. The README points at PyMuPDF's licensing page for its table detection functionality, and PyMuPDF is not MIT. It also names unitable and Microsoft's Table Transformer as ml backends, each with its own licence terms. If you plan to ship a commercial product using the ml table path or PyMuPDF's detection, read those licences yourself; this is a description of what the README links to, not legal advice. The core parser built on pdfminer.six is the path with the fewest licence questions.
Who should pick this up, and what to check first
open-parse fits teams already committed to Python, already running RAG over PDFs, and already dissatisfied with token-count chunking. The two-stage design is the reason to choose it: you get layout-aware nodes you can serialize, and you can decide separately whether to cluster them semantically. The reason to hesitate is operational. The TESSDATA_PREFIX requirement is not optional for scanned documents, the ml table path adds a model download step, and the semantic pipeline adds an embedding API dependency. Before adopting, run the basic example against one representative document from your own corpus and inspect the nodes, because the quality of grouping depends on how well the parser segments your particular layouts. If the nodes look right, then test the semantic pipeline with your own min_tokens and max_tokens values, since the README's 64 and 1024 are sample settings rather than defaults derived from your content. If the nodes look wrong, no amount of clustering will fix it, and a layout parser plus your own grouping code may serve you better.
Editorial conclusion
Adopt open-parse if your RAG pipeline ingests structured PDFs and you want chunk boundaries that follow headings, sections and tables rather than token counts. Do not adopt it if you cannot install Tesseract for scanned documents, or if you need a guaranteed offline parser with no embedding API calls. Before committing, verify the TESSDATA_PREFIX path on your target OS, check whether pip install "openparse[ml]" and openparse-download succeed in your environment, and confirm that the node output shape from parsed_content.dict() matches what your downstream index expects.
Community notes