Unlimited-OCR: One-Pass Parsing for Long Documents, With Real Deployment Choices

What It Solves and Who It Is For

Traditional OCR pipelines split a page into chunks, run detection and recognition separately, then stitch the results back together. That fails on long documents where layout and reading order matter. Unlimited-OCR, from Baidu, is a model that parses an entire long document or image in one forward pass, producing structured text with detected regions and categories. It is aimed at engineers who need to feed scanned books, multi-page contracts, or dense academic papers into downstream AI workflows, where a single coherent output beats a pile of fragmented text blocks. The README positions it as pushing DeepSeek-OCR further, so the intended audience is teams already working with vision-language models for OCR, not casual users who just want a quick screenshot-to-text tool.

The One-Pass Mechanism and Its Two Image Modes

The core idea is that the model takes an image or a PDF converted to images and outputs text with structural markers. The README shows a prompt like '<image>document parsin' (truncated), implying the model is conditioned on the image directly. It supports two configurations for a single image: 'gundam' and 'base'. 'gundam' uses base_size=1024, image_size=640, and crop_mode=True, which means it crops the image into smaller tiles for higher resolution on text-dense regions. 'base' uses base_size=1024, image_size=1024, and crop_mode=False, which processes the whole image at once. The trade-off is clear: 'gundam' likely gives better accuracy on small text but may lose global context, while 'base' preserves layout but may miss fine details. The model also outputs special markers like '<|det|>type [bbox]<|/det|>' to indicate detected regions and their bounding boxes. That is not just decoration; you need those markers to reconstruct blocks in post-processing.

Getting It Running: Transformers, vLLM, or SGLang

You have three paths. The quickest is Hugging Face Transformers. You need Python 3.12.3, CUDA 12.9, and specific package versions: torch==2.10.0, torchvision==0.25.0, transformers==4.57.1, Pillow==12.1.1, pymupdf==1.27.2.2, and a few others. The code snippet is straightforward: load the model with trust_remote_code=True, use bfloat16, and call model.infer(tokenizer, prompt='<image>document parsin'). That will work on a single GPU, but the README does not state the minimum VRAM. The second path is vLLM, which requires a Docker image: vllm/vllm-openai:unlimited-ocr for CUDA 13.0, or vllm/vllm-openai:unlimited-ocr-cu129 for Hopper GPUs. The third path is SGLang, which is more involved. You create a uv virtualenv, install a local SGLang wheel, pin kernels==0.11.7, and install pymupdf. Then you launch the server with specific flags: --attention-backend fa3, --page-size 1, --context-length 32768, --enable-custom-logit-processor, and --disable-overlap-schedule. Each path has its own quirks, so choose based on your existing infrastructure.

The SGLang Server and Batch Inference Script

For production use, the SGLang server is the most interesting. It exposes an OpenAI-compatible API, so you can send base64-encoded images or PDF pages as requests. The README shows a Python client that converts PDF pages to images using PyMuPDF at 300 DPI, then sends them to the server. There is also a batch script, infer.py, that starts the server automatically and sends concurrent requests for an image directory or a PDF. You can set concurrency (example uses 8), choose the image mode with --image_mode gundam, and specify the model directory or GPU ID. That script is a practical entry point for processing many documents without writing your own client. However, the script is not a full pipeline. It outputs raw text with markers, not clean structured JSON. You still need to write post-processing logic to turn those markers into usable blocks.

The Post-Processing Requirement You Cannot Ignore

The README includes a specific function, remove_det, for OmniDocBench evaluation. That function uses a regex to strip '<|det|>type [bbox]<|/det|>' markers and group lines into blocks. This is not optional cleanup. The model outputs markers that indicate categories like 'image' or text, and without removing them, your downstream text will be full of noise. The function groups lines belonging to the same block with a single newline and separates blocks with double newlines. That gives you a rough structure, but it is not a full parser. If you need to know whether a block is a table, a figure, or a heading, you have to extend this logic yourself. The README only shows the pattern for 'image' category, so you will need to inspect the actual output to handle other types. This is a real maintenance cost that the project does not fully cover.

Limitations and Failure Modes

The most obvious limitation is the context length. The SGLang server is launched with --context-length 32768, which caps how much text the model can produce in one pass. For a very long PDF, you may hit that limit and get truncated output. The README does not explain what happens in that case, so you must test it. Another limitation is GPU memory. The model is large, and the 'gundam' mode crops images, which might increase memory usage per request. The README does not list minimum VRAM, but running a 32k-context model with bfloat16 on a single GPU will likely require 24GB or more. The 'base' mode processes 1024x1024 images without cropping, which could be faster but less accurate on dense text. Also, the project has no tagged releases, and the README references a paper (arXiv 2606.23050) that may not be widely available. If you need stability, this is a moving target.

Alternatives and How They Differ

The README explicitly acknowledges DeepSeek-OCR and DeepSeek-OCR-2 as inspirations. DeepSeek-OCR is a similar vision-language model for OCR, but it is not designed for long-horizon parsing in one pass. It typically processes individual pages or images, so for a multi-page document you have to stitch results yourself. Unlimited-OCR aims to handle the whole document at once, which is the key difference. PaddleOCR, also from Baidu, is a traditional OCR toolkit that does detection and recognition separately, with a focus on speed and many languages. It is far lighter and runs on CPU, but it does not produce structured, context-aware text. If you need simple text extraction from scanned documents, PaddleOCR is a better fit. If you need a model that understands layout and can parse a whole book, Unlimited-OCR is the direction, but you pay for it with heavy dependencies and GPU requirements.

Maintenance, Upgrade Cost, and License

The project is under the MIT license, which is permissive and allows commercial use with attribution. That is a plus for adoption. However, maintenance is a concern. The README lists no releases, and the last push date is unknown. The model is available on Hugging Face, ModelScope, and Baidu Cloud, which suggests active support, but the codebase is tied to specific versions of torch, transformers, and SGLang. Upgrading any of those may break the trust_remote_code logic or the custom logit processor. The dependency on a local SGLang wheel (sglang-0.0.0.dev11416) is particularly fragile; that is a development version, not a stable release. If you use vLLM, you are pinned to a specific Docker image, which is easier to manage but still requires pulling a large image. Plan for regular re-testing whenever you update your stack. The paper citation is from 2026, so this is a recent project, and the API may change.

Editorial conclusion

Adopt Unlimited-OCR if you need to parse entire long documents in one pass and can afford a large GPU with at least 32GB memory, especially if you already use vLLM or SGLang for serving. Do not use it if your documents are short, your hardware is modest, or you cannot handle the required post-processing to strip detection markers. Before committing, verify the exact context length limit, test the 'gundam' vs 'base' image modes on your own PDFs, and confirm that the SGLang wheel and kernel versions match your CUDA environment. The project is young, with no tagged releases, so plan for breaking changes.

Official sources

1. Official README
2. Project repository