OnnxOCR: local OCR and document processing with ONNXRuntime

Why OnnxOCR removes the PaddlePaddle training dependency

OnnxOCR targets developers who need OCR inference in Python applications but do not want the application stack tied to PaddlePaddle, the framework used to train the originating PaddleOCR models. The repository describes itself as based on PaddleOCR and decoupled from PaddlePaddle deep learning training. That distinction matters for deployment planning: the models remain PaddleOCR-derived ONNX files, while application code executes them through ONNXRuntime.

The intended users include engineers building OCR services, local document tools, license plate workflows, and structured extraction pipelines. The documented PP-OCRv5 general OCR model covers Simplified Chinese, Traditional Chinese, Chinese Pinyin, English, and Japanese. The package also exposes narrower workloads for Chinese and English layout analysis, tables, license plates, and Markdown-oriented document processing.

This is not a model training framework. The README provides no training commands, dataset preparation instructions, or PaddlePaddle training examples. The project should therefore be evaluated as an inference and application layer, not as a replacement for PaddleOCR training. The latest documented release, v3.0.0 dated 2026-05-27, also added a separate Qwen3.5-2B extraction workflow, extending the project beyond recognition into field extraction from OCR output.

The ONNXRuntime session path across OnnxOCR models

The central technical choice is a unified inference engine. The README states that all ONNX models create ONNXRuntime sessions through `onnxocr/inference_engine.py`. That gives different recognition and analysis components a common runtime boundary. It also leaves a specific extension point for vendors: the documentation says GPU and NPU providers can be adapted by modifying that unified engine.

The general OCR path accepts an image and produces recognized text associated with detection results. The repository includes `test_ocr.py` for this broad entry point and `tests/test_general_ocr.py` for the bundled PP-OCRv5 path. Optional feature tests show that the package is organized around separate workloads rather than one indivisible OCR function. License plate recognition has its own test, as do tables, layout analysis, and layout Markdown generation.

The Qwen workflow adds a second data path. OnnxOCR first recognizes full-page text and text boxes, then passes the OCR text into a locally stored Qwen3.5-2B ONNX model. The extraction stage can return structured JSON or fields defined by an application template. The end-to-end example described in the release notes is an identity-card workflow: OCR supplies the text, and the language model extracts fields. This separation allows an engineer to replace the extraction template without changing recognition code.

The repository describes the runtime as supporting ARM and x86 devices. It does not provide measured latency, memory use, or device-specific provider configuration in the available documentation, so architecture compatibility should not be treated as proof of equal performance.

Installing OnnxOCR and running its bundled OCR test

The README specifies Python 3.8 or later and installation from `requirements.txt`. The first command uses the Tsinghua PyPI mirror as shown in the project documentation.

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt

The dependency file requires `onnxruntime>=1.26.0` and pins `numpy<2.0.0`. It also includes Flask and Gunicorn for the service layer, OpenCV packages for image work, model-hosting clients, and document libraries including `pymupdf`, `pdf2image`, `pypdfium2`, `pypdf`, `pdfminer.six`, and `pdftext`. These dependencies make the general install broader than an ONNX inference-only stack.

For a first local use, run the repository's main OCR example after installation:

python test_ocr.py

The README says this script runs general OCR by default. Optional examples are present but commented out, which prevents the default path from silently selecting features whose model files may not have been downloaded. The bundled PP-OCRv5 general OCR files are the repository's documented default model set.

A feature-focused check is also provided:

python tests/test_general_ocr.py

This is the first useful validation for a general OCR adoption. Optional feature tests should be attempted only after their corresponding models are available. The README describes output files and visual results, but the truncated documentation does not specify a stable command-line output schema, so application code should use the package API rather than assume undocumented filenames.

Downloading optional OnnxOCR models before using extra endpoints

A checkout does not contain every model required by the advertised feature set. The README says license plate, table, layout, orientation, and RapidDoc Markdown models are large and are downloaded on demand. International users are directed to the HuggingFace repository `jingsongliu/onnxocr_model`, while the default model downloader uses ModelScope.

The documented HuggingFace download command is:

python scripts/download_models.py --source huggingface

The script copies a repository `models/` directory into local `onnxocr/models/` and checks required optional files, including `onnxocr/models/rapid_doc/layout/pp_doclayoutv2.onnx`. Users in mainland China are directed to run the command without that source option:

python scripts/download_models.py

That command draws from the ModelScope repository `supersong/onnxocr_model`. An application can also use `huggingface_hub.snapshot_download` directly, as shown by the README:

from huggingface_hub import snapshot_download

model_dir = snapshot_download("jingsongliu/onnxocr_model")

Before starting downloads, the project provides a local verification mode:

python scripts/download_models.py --check-only

This check is particularly relevant for container images and deployment automation. Feature names in the package do not imply that their model artifacts are bundled. The added `/plate`, `/table`, `/layout`, and `/layout_markdown` HTTP endpoints also require the corresponding models and service code to be present.

Running OnnxOCR through its Flask container service

The repository includes `app-service.py`, `webui.py`, templates, static assets, Gunicorn, and Flask. Its `Dockerfile` provides the clearest documented service deployment path. The image uses `python:3.10-slim`, installs FFmpeg and several shared libraries, copies the repository into `/app`, and exposes port 5005.

The image's startup command is documented directly in the `Dockerfile`:

CMD ["python", "app-service.py"]

The container sets `ONNXOCR_PORT=5005` and exposes the same port. A host port mapping still has to be supplied when the container is started, and the README excerpt does not provide a complete `docker run` example. Likewise, it does not enumerate every request body, response schema, authentication rule, or endpoint parameter for `/plate`, `/table`, `/layout`, and `/layout_markdown`.

This makes the container useful as an execution environment but leaves API integration dependent on the repository's route definitions. The service should not be exposed as a finished public endpoint based on this excerpt alone. In particular, the available documentation does not establish an authentication mechanism, request-size limit, concurrency setting, or production readiness beyond Gunicorn being present in the dependency list.

Qwen3.5-2B extraction as a separate OnnxOCR model boundary

The v3.0.0 workflow separates OCR from local information extraction. Qwen3.5-2B ONNX files come from a dedicated ModelScope repository and are stored under `onnxocr/models/qwen_2b`. Download the documented quantized variant and verify it with the project script:

python examples/qwen35_2b_onnx.py download --variant q4
python examples/qwen35_2b_onnx.py verify

The README provides two smoke tests. The first generates text from a prompt:

python examples/qwen35_2b_onnx.py run-python --prompt "Hello, introduce yourself briefly."

The second combines an image with a prompt:

python examples/qwen35_2b_onnx.py run-python --image onnxocr/models/qwen_2b/images/demo.jpeg --prompt "Describe this image in one short sentence."

Package-level use creates `Qwen35ONNX` with a local model directory. Its `generate` method returns generated text and a stop state:

from onnxocr.qwen35_2b import Qwen35ONNX

qwen = Qwen35ONNX("onnxocr/models/qwen_2b")
text, stopped = qwen.generate("Extract key fields from this OCR text: ...")
print(text)

The important constraint is that the base package does not include vertical-scene prompts. Applications must define their own `ExtractionTemplate` fields and rules. That is a clear division of responsibility, but it also means extraction quality depends on both OCR output and the application's prompt and validation design. The README does not document rollback or an automatic compatibility policy between auxiliary model versions.

OnnxOCR licensing, maintenance, and adoption limits

OnnxOCR is distributed under Apache-2.0. That is the repository's stated license, but adopting it still requires reviewing the license text for the project itself and checking the terms attached separately to downloaded models, sample images, and model-hosting repositories. This article does not provide legal advice or infer rights beyond the declared license.

The repository is not archived, and the last push was on 2026-06-11. The latest listed release is v3.0.0, dated 2026-05-27. These facts indicate recent project activity, but they do not document a long-term compatibility policy, a stable public API commitment, or the cost of migrating between model generations.

OnnxOCR is the wrong tool when its optional model downloads are unacceptable or when the project must remain limited to the bundled general OCR files. It is also a weak choice for developers seeking model training support. The documented project is an ONNX inference and application layer, with additional document workflows. The dependencies include `numpy<2.0.0`, which can affect integration with an existing Python environment that requires NumPy 2.x. The service documentation also leaves security, API validation, and production deployment settings unstated in the available excerpt.

For teams already operating ONNXRuntime, the unified `onnxocr/inference_engine.py` boundary is the central architectural reason to consider adoption. For teams beginning with OCR research, model training, or accuracy evaluation, PaddleOCR offers a broader documented context, while OnnxOCR concentrates on deployment through ONNX models.

Editorial conclusion

OnnxOCR fits teams that already have ONNX models or want PaddleOCR-style inference without a PaddlePaddle training runtime, especially for local document-processing pipelines. It is a poor fit for projects that require the included models alone to cover every optional feature, because several capabilities require separate downloads and feature-specific tests. Before adoption, check the required model files, runtime compatibility for the target CPU architecture, and the documented test for the exact OCR or document workflow. For general OCR, the strongest evidence is that `tests/test_general_ocr.py` uses bundled PP-OCRv5 files; optional workloads still depend on files fetched from HuggingFace or ModelScope.

Frequently asked questions

Official sources

1. Issues
2. jingsongliujing/OnnxOCR on GitHub
3. License: Apache-2.0
4. README
5. Releases