MonkeyOCRv2: a document-native vision backbone for OCR and parsing

One encoder, many document tasks

Most OCR systems are pipelines assembled from parts. MonkeyOCRv2, presented as a visual-text foundation model for document AI, leads with a different unit: a standalone, document-native vision encoder that other systems can integrate as a visual backbone. The current release has been evaluated, per the README, on document parsing, document understanding, text recognition, formula recognition, text detection, document tampering detection and overlapping-text segmentation.

The project also keeps a longer list in view. Beyond the evaluated tasks, the README suggests the encoder may suit text-rich scenarios such as scientific papers, historical documents, medical reports, charts and tables, and remote-sensing maps or reports, and explicitly invites community exploration of those directions. The wording is careful: suggested, evaluated, released are three different claims, and the README keeps them separate.

The model zoo, from 21M to parsing pipelines

The zoo splits by job. The vision encoders come in three sizes: MonkeyOCRv2-S on a ViT-S backbone at 28 million parameters trained at 1280 by 28 by 28 resolution, MonkeyOCRv2-B on ViT-B at 113 million parameters at the same resolution, and MonkeyOCRv2-AS on a ViTAEv2-S backbone at 21 million parameters at 1760 by 32 by 32, with the first two listed for recognition, parsing and understanding and the third for detection and segmentation.

The parsing models combine the encoder with a language model: the README's table lists MonkeyOCRv2-S-Parsing at 0.6 billion total parameters, of which the ViT is 0.03 billion and the LLM 0.6 billion, with per-language scores across seventeen languages in digital-born and photographed categories. A separate variant, MonkeyOCRv2-B-Parsing-DFlash, released on 2026-07-24, enables vLLM serving with DFlash for what the project reports as up to two times faster inference.

Using the encoder in your own stack

The README's integration example is deliberately small, a transformers load of the base encoder:

from transformers import AutoModel

encoder = AutoModel.from_pretrained(
    "zenosai/MonkeyOCRv2-B",
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)

Note the trust_remote_code flag: the checkpoint executes model code from its repository, which is normal for custom architectures and still worth knowing before you load one in a shared environment. From there, the README points to the vision encoder quick start section for installation and feature-extraction examples, and invites issues or pull requests if you adapt the encoder to a new task or domain.

The repository tree shows the task coverage as directories: detection, recognition, parsing, formula, forensics, understanding and vision, plus a download_model.py script and an images_test directory, so each task family carries its own code rather than one monolithic pipeline.

MonkeyDoc v2: the dataset behind the claims

Alongside the models, the project released MonkeyDoc v2, described as currently the largest document image pretraining image-text pair dataset, at 113 million document images across 17 languages. The open-sourcing is still underway: so far 52 million synthetic samples and 52 million real-world samples have been released.

The README is candid about the cost of taking the whole thing. Downloading the processed and compressed dataset currently requires approximately 10 TB of disk space, and the project recommends at least 11 TB available before starting. The download runs through ModelScope:

pip install modelscope
modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2

For teams training document models rather than just running them, this corpus is arguably the more valuable release: the README frames it as a shared data foundation for more transparent, reproducible and fair comparisons in document AI, which is a dig at papers trained on undisclosed data.

Benchmarks, integrations and the CPU path

The news log tracks the project's momentum. On 2026-07-22 the project reported MonkeyOCRv2-B-Parsing ranking first among evaluated open-source models on the official MDPBench leaderboard, with an 83.3 overall score across 17 languages covering both digital-born and photographed documents; that is the project's own report of a leaderboard result, with the leaderboard itself linked. On 2026-09-07, RAGFlow added MonkeyOCRv2-Parsing as a PDF parser, which matters because it makes the model reachable from an established retrieval pipeline without custom glue.

Practical operability got attention too. CPU-only support for MonkeyOCRv2-Parsing arrived on 2026-08-22 with instructions in docs/cpu_support.md, and on 2026-08-17 the project published training and evaluation instructions for recognition, detection, overlapping-text segmentation and formula recognition, which is what turns released checkpoints into reproducible baselines. A hosted demo is linked from the README for a first look without any installation.

Licence and adoption cautions

GitHub reports the licence as NOASSERTION, meaning a custom licence file ships with the repository; read it before production use, since foundation-model licences often carry conditions that permissive templates do not. The README's scores and rankings are the project's own reports, and while the MDPBench leaderboard is external, the per-language table in the README has no third-party audit attached.

Two adoption notes follow from the structure. First, the parsing models are small by current standards, a 0.6 billion parameter LLM paired with a 0.03 billion parameter ViT in the S variant, which is exactly why CPU parsing is plausible; whether that accuracy holds on your documents is what the demo and your own evaluation are for. Second, the last push was on 2026-09-15 and the project is moving quickly, so pin the checkpoint versions you validate against.

Editorial conclusion

MonkeyOCRv2 fits teams building document AI that want a compact, document-native vision encoder to embed in their own pipelines, and teams that need multilingual parsing with a small enough model to run without a GPU, thanks to the documented CPU path. It does not fit readers who need independently audited accuracy claims, since the benchmarks here are the project's own reports, and it does not fit anyone unwilling to read a custom licence before production use. Before adopting, try the hosted demo on your own documents, load the base encoder with the transformers example, and pin checkpoint versions, since the last push was on 2026-09-15 and the zoo is still growing.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Yuliang-Liu/MonkeyOCRv2 on GitHub