docext: OCR-free document extraction that runs on your own hardware

The gap docext is aimed at: extraction without a text layer and without a cloud call

Most document extraction pipelines assume a text layer. If a PDF was produced by a scanner, a fax machine, or an export tool that embedded glyphs as images, that assumption fails and you fall back to OCR. docext takes the other route. The README describes it as an on-premises toolkit powered by vision-language models, and the three capabilities it lists (PDF and image to markdown, structured information extraction, and the Intelligent Document Processing Leaderboard) all sit on top of a VLM rather than a classical OCR engine.

The intended user is an engineer who has documents that cannot be sent to a third-party API, either because of data residency rules or because the volume makes per-page pricing unattractive. The README states on-premises deployment runs on Linux and macOS, and that a REST API is available for programmatic access. That combination, a local model plus an HTTP surface, is the shape of a self-hosted service rather than a library you import and forget.

There is a second audience that is easy to miss. The repository also carries the benchmark code behind idp-leaderboard.org, which evaluates models across seven tasks: key information extraction, visual question answering, OCR, document classification, long document processing, table extraction, and confidence score calibration. If your job is choosing a VLM for document work rather than shipping a pipeline, that half of the repository is the relevant half.

How the three parts fit together, and why they are documented separately

The repository is effectively three products sharing one package name. The README points to three separate guides: PDF2MD_README.md for markdown conversion, EXT_README.md for extraction, and docext/benchmark for the leaderboard. That structure is a signal. A user who wants markdown output is not necessarily the same user who wants invoice fields, and the two paths do not appear to share a configuration file.

The markdown path is the most concretely specified. According to the README, conversion recognizes LaTeX equations in both inline and block form, generates descriptions for images inside img tags, and marks signatures, watermarks and page numbers inside signature, watermark and page_number tags respectively. Checkboxes and radio buttons become the Unicode characters for empty, checked and crossed boxes. Tables are emitted as HTML tables rather than markdown pipe tables, which is a deliberate choice: HTML tables can express row and column spans that markdown cannot.

The extraction path works differently. You define custom fields or start from the pre-built invoice and passport templates, and the output carries confidence levels for extracted information. The README also mentions adding or deleting fields and columns to adapt a template for other document types, which suggests templates are data rather than code.

The benchmark path is a harness. It runs models against the seven task categories and produces the numbers that appear on the live leaderboard. The changelog shows the cadence: entries were added for gemma-3-27b-it, InternVL3-38B-Instruct, qwen2.5-vl-32b-instruct, claude-sonnet-4, mistral-medium-3 and gemini-2.5-pro-preview variants across May and June 2025. That is a benchmark maintained by its authors, which is worth noting when you read the rankings.

Getting it running: install, then pick a guide

The README does not inline the installation commands. It directs readers to the feature guides, so the first real step is choosing which of the three you need and opening the matching document. The package is published on PyPI as docext, which is what the PyPI version badge in the README points at, so a pip install of that name is the expected entry point, but confirm the exact command in the guide rather than assuming flags.

Configuration differs by path. For markdown conversion, the PDF2MD_README.md guide covers setup and the feature list above. For extraction, EXT_README.md covers installation and usage, and the extraction surface is where the template mechanism lives: you either select a pre-built template such as invoice or passport, or define custom fields, and you can add or delete fields and columns. The REST API is described as the integration point for applications, which means the extraction service is meant to run as a process you call over HTTP rather than as an in-process function.

One concrete constraint from the README is worth flagging before you start: on-premises deployment is listed for Linux and macOS. Windows is not mentioned. If your infrastructure is Windows-based, that is a gap you will have to work around, and the README does not describe one.

For the benchmark, the guide lives in the docext/benchmark directory of the repository. Because it evaluates models rather than documents, its setup will differ from the extraction path, and the README treats it as a third, separate track.

The model dependency is the real deployment cost

Nothing in docext is self-contained. Every capability routes through a vision-language model, and the repository does not ship one. The README's own announcement of Nanonets-OCR-s describes it as a compact 3B parameter model trained for image to markdown conversion, and links to a Hugging Face page rather than bundling weights. That is the honest reading of the architecture: docext is the orchestration, prompting and post-processing layer, and you supply the inference engine.

This has consequences the README does not spell out. Your throughput ceiling is your GPU, not the library. Memory requirements scale with the checkpoint you choose, and the choice is not free: a 3B model is described as compact and efficient, while the leaderboard tracks much larger entries such as InternVL3-38B-Instruct and qwen2.5-vl-32b-instruct. There is a direct trade between the hardware you own and the accuracy you get, and docext's leaderboard is the tool it gives you to reason about that trade rather than a promise that resolves it.

The second consequence is that output stability depends on the model, not on docext. Semantic tagging (signature, watermark, page_number, img) is a prompting contract. A model that drifts from the expected tag format will produce markdown that parses differently. The README presents these tags as features without describing validation or repair, so treat tag conformance as something you test on your own documents rather than something the library guarantees.

Where docext is the wrong tool

If your PDFs have a clean text layer, docext is overkill. Running a vision-language model over a digitally generated contract to recover text that is already selectable adds GPU cost and a failure mode (hallucinated content) that a deterministic text extractor does not have. The README positions the toolkit around vision-language models, and that positioning is the limitation: it is built for the cases where text extraction alone fails.

Confidence scoring deserves scrutiny too. The README lists confidence levels for extracted information and separately lists confidence score calibration as one of the seven benchmark categories. The existence of a calibration benchmark is a hint that confidence numbers are not automatically trustworthy across models. The README does not state that docext calibrates scores, only that it returns them. If you plan to route low-confidence documents to human review, the threshold is yours to validate, not the library's to supply.

Table extraction has a similar boundary. The markdown path emits HTML tables, which is more expressive than pipe tables but also means downstream consumers must handle HTML rather than plain markdown. If your pipeline expects markdown tables, you will be converting.

Finally, the release history is sparse and recent. The listed releases are v0.1.2, v0.1.7 and v0.1.14, spanning April to June 2025, with a repository push in March 2026. That is a young version line. Pre-1.0 versioning means interfaces can move, and the three-guide documentation split means a change in one path may not be reflected in the others.

Alternatives and the actual difference in approach

The most direct comparison is a conventional OCR engine feeding a rules or ML extraction layer. Tesseract plus a template matcher is deterministic, runs on CPU, and produces the same output for the same input every time. docext trades that determinism for flexibility: it can read a signature, describe an image, convert a LaTeX equation and handle a scanned form without you writing layout-specific rules. The cost is a GPU, a model dependency, and non-deterministic output.

The second comparison is a hosted document AI API, including the one Nanonets itself sells. The README's About section links to Nanonets' commercial document parsing product, so the vendor is explicit that docext is the open-source, self-hosted counterpart. The difference is operational, not algorithmic: a hosted API gives you someone else's GPUs, uptime and model upgrades, and takes your documents off your premises. docext inverts all three. If data residency is the binding constraint, that inversion is the whole point.

The third comparison is the benchmark itself against other document AI leaderboards. docext's differentiator there is scope: seven task categories including confidence score calibration and long document processing, which many OCR-focused benchmarks skip. If you are selecting a VLM and your documents are long or your pipeline depends on confidence thresholds, those two categories are the ones that matter and the ones usually missing elsewhere.

Licence and the cost of staying current

docext is Apache-2.0, which permits commercial use, modification and redistribution provided you preserve the licence and notices. That is the permissive end of the spectrum, and it means embedding docext in a commercial product does not by itself trigger a copyleft obligation. This is a statement about the licence text, not legal advice; if you are redistributing a modified version or combining it with differently licensed components, have counsel review the specifics.

The maintenance cost sits mostly outside the library. Model checkpoints move faster than the toolkit: the changelog shows leaderboard entries added for new model versions roughly weekly through May and June 2025. Staying current means re-evaluating on your own documents each time a checkpoint you depend on is superseded, because the leaderboard ranks models on the benchmark's data, not yours. docext gives you the harness to do that, but the work is yours.

Upgrading docext itself carries the usual pre-1.0 risk. With three separate guides and three capability areas, a version bump can change one path without touching the others, so pin the version you validate against and re-run your own extraction fixtures on upgrade rather than relying on the changelog, which in the README documents leaderboard additions more thoroughly than library changes.

Editorial conclusion

Adopt docext if you need document extraction that never leaves your infrastructure and you are willing to run a vision-language model yourself on Linux or macOS. Do not adopt it if you want a managed pipeline with SLAs, or if your documents are short, plain text PDFs where a conventional text-layer parser is cheaper and more predictable. Before committing, verify three things: which VLM checkpoint your hardware can actually host, whether the pre-built invoice and passport templates match your field names, and how the confidence scores behave on your own documents, since the README describes scoring but does not state a calibration guarantee.

Official sources

1. License: Apache-2.0
2. NanoNets/docext on GitHub
3. Project website
4. README
5. Releases