TurboOCR: a TensorRT document parser behind HTTP and gRPC

What TurboOCR replaces, and for whom

Most OCR stacks answer one question: what text is in this image. TurboOCR answers a larger one. The README describes it as a document parser rather than an OCR engine, covering text detection and recognition, layout, tables converted to HTML, formulas converted to LaTeX, and reading-order Markdown. That scope matters because the people who need it are usually building ingestion pipelines for search or retrieval, not reading a single scan by hand. If your input is a folder of receipts and forms and your output is structured text, the extra stages save you from stitching three separate models together. The project targets Linux with an NVIDIA GPU as the only shipped backend, and the README is explicit that Apple Metal and Intel OpenVINO are in testing while AMD ROCm is in development. The stated requirement is driver 595 or newer and a Turing or newer GPU, so GTX 16-series and RTX 20-series cards are the floor.

The single-engine pipeline and how requests flow

The architecture is one multi-stream CUDA/TensorRT engine rather than a chain of Python processes. Detection and recognition come from PP-OCRv6, and the README says one model covers Latin, Chinese and Japanese, with Arabic, Cyrillic, Korean, Thai and Greek handled by retained PP-OCRv5 recognizers. Layout uses PP-DocLayoutV3 with 25 classes and a class-aware XY-cut for reading order. Tables go through SLANet+ and formulas through PP-FormulaNet-S. The important design point is that these stages are opt-in per request rather than always-on. Layout is on by default, but tables and formulas only run when a request asks for them, and asking for either auto-enables layout. That keeps the cheap path cheap: a plain text extraction does not pay for table structure recovery. The README also notes the whole thing runs locally with no VLM, which is a deliberate contrast with vision-language document parsers. Requests arrive at nginx on port 8000, which reverse-proxies to Drogon on port 8080, and gRPC is exposed on 50051.

Starting the container and choosing stages

The README gives a one-line Docker start: docker run --gpus all -p 8000:8000 -p 50051:50051 -v trt-cache:/home/ocr/.cache/turbo-ocr ghcr.io/aiptimizer/turboocr:latest. The named volume matters. On first start the container builds TensorRT engines from ONNX, which the README says takes about 90 seconds on a 5090 and up to an hour on older cards. The volume caches those engines so later starts are instant. During that build, requests get a connection refused error from nginx until the backend is ready, which is worth knowing before you wire it into a health check. TRT_OPT_LEVEL=3 is documented as cutting build time 3 to 5 times with a small speed regression. Stages are selected with environment variables rather than file paths, because the weights are baked into the image: TABLE_BACKEND=slanext for tables, FORMULA_BACKEND=ppformulanet_s for formulas, and OCR_MODEL set to tiny, small, medium, arabic, eslav, korean, thai or greek. The README notes tiny is the default. VRAM planning is stated as roughly 4 GB for text-only and 8 GB for the full pipeline, with each extra PIPELINE_POOL_SIZE replica adding another full set.

The endpoints and the flags that turn stages on

The API is small enough to describe in a paragraph. POST /ocr/raw takes an image body and returns a results array with text, confidence and a four-point bounding_box. Query parameters control the pipeline: layout=1, tables=1, formulas=1, combinable, with tables and formulas implying layout. PDFs go to POST /ocr/pdf, and adding markdown=1 to that endpoint returns Markdown. A single page to Markdown is POST /ocr/markdown. gRPC is on port 50051. The README also mentions GET /capabilities, which reports which stages a running server has loaded, and Prometheus metrics on /metrics. That capabilities endpoint is the practical way to confirm that the container you started actually loaded the backends you asked for, since a missing TABLE_BACKEND would otherwise only show up as absent output on a real document.

Where the throughput numbers come from, and what they measure

The headline figures are up to 559 images/s on receipts, 520 on forms, and 200-plus on dense documents, all on one RTX 5090. Full structured parsing with layout, tables and formulas is quoted at about 20 pages/s. The README frames these against VLM document parsers such as PaddleOCR-VL at roughly 1 page/s, and against classic engines at 15 to 90 times slower on forms and receipts. Two caveats belong next to those numbers. First, they are the project's own benchmark section, and the repository description itself cites over 200 images/s on OmnidocBench, which is a different and lower figure than the receipts peak. Second, the README's own accuracy claim is narrower than its speed claim: it says TurboOCR is accurate on forms and receipts, and competitive with PaddleOCR-VL, PaddleOCR-Python, RapidOCR, EasyOCR and Tesseract. Dense-document accuracy is not given the same confidence in the text. Treat the speed figures as a property of the receipt and form workload, and validate dense pages yourself.

The GPU lock-in and the first-start cost

The clearest limitation is hardware. NVIDIA is the only backend shipped today, so a CPU-only deployment, a Mac, or an AMD card is out of scope for now, and the README's own banner says so. The second is the engine build. A cold start on an older Turing card can take up to an hour before the service answers, and the container returns connection refused during that window rather than a clear not-ready status. If your deployment model is ephemeral containers with no persistent volume, you pay that build on every start unless you mount the cache. The third is memory. The full pipeline wants around 8 GB, and raising PIPELINE_POOL_SIZE for concurrency multiplies that per replica, so on a smaller card you are choosing between concurrency and stage coverage. The fourth is scope: this is not a handwriting recognizer, a general vision-language model, or a PDF text-layer extractor. It renders and OCRs pages. If your PDFs already carry a clean text layer, running a GPU parser over them is wasted work.

How it differs from PaddleOCR-Python and PaddleOCR-VL

The README positions TurboOCR against two different kinds of alternative. PaddleOCR-Python is the same model family run through a Python stack, and the claimed difference is throughput on forms and receipts plus a single compiled engine instead of a Python inference path. PaddleOCR-VL is a vision-language document parser, and the difference there is architectural: TurboOCR runs detection, recognition, layout, tables and formulas as separate specialized models on one TensorRT engine with no VLM in the loop, while a VLM reads the page more holistically at roughly 1 page/s by the README's figure. That trade is real in both directions. The VLM approach tends to handle unusual layouts and messy real-world pages with less per-stage tuning, and it produces Markdown more directly. TurboOCR wins when the page types are known and the volume is high, which is why the project's own accuracy language is careful to say forms and receipts. If your corpus is heterogeneous scans of unknown provenance, the speed advantage is only useful after you have established that the per-stage pipeline holds up on it.

Licence, maintenance and upgrade cost

TurboOCR is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are included. That is a permissive licence with no copyleft obligation on your own code, but it also means no warranty and no support commitment from the authors. The MIT terms do not cover the model weights or the CUDA, TensorRT and PaddleOCR components you pull in alongside it, so check those separately; this is a description of the licence text, not legal advice. On maintenance, the release cadence visible in the supplied material is rapid: v3.5.3, v3.5.5 and v3.5.6 all landed within four days in September 2026, and the release notes for those versions mention bounded memory, removal of silent fallbacks, aarch64 builds and OpenCV 5. Rapid point releases are a sign of active work, and also a sign that pinning a digest rather than using latest is wise. The v3.0 upgrade notes are flagged as containing breaking changes, and the model tier system with tiny, small and medium is new in that line, so an upgrade from v2 is a migration rather than a drop-in. Budget for re-running your own accuracy checks after each tier or model change, because the recognizer you were relying on may not be the default one after an upgrade.

Editorial conclusion

Adopt TurboOCR if you already run Linux with a Turing-or-newer NVIDIA card and you need high-volume OCR plus optional layout, table and formula extraction behind an HTTP or gRPC endpoint. Skip it if you need CPU-only inference, a Mac or Windows host, or a backend other than NVIDIA today, since the README states Apple Metal and Intel OpenVINO are still in testing and AMD ROCm is in development. Before committing, verify on your own hardware that the first-start TensorRT engine build finishes in a tolerable window, that your card has the roughly 4 GB text-only or 8 GB full-pipeline VRAM, and that the per-request flags you need are reported by GET /capabilities on the running container.

Official sources

1. aiptimizer/TurboOCR on GitHub
2. License: MIT
3. Project website
4. README
5. Releases