onnx-tensorrt: the parser between an ONNX graph and a TensorRT engine
ONNX-TensorRT: TensorRT backend for ONNX
At a glance
- What is it?
- onnx-tensorrt is the C++ parser library that reads an ONNX model and hands TensorRT a network definition it can build. It is version-locked to a TensorRT release, ships as libnvonnxparser, and is the layer you touch when a model parses but will not build.
- Who is it for?
- Adopt onnx-tensorrt if you are already pinned to a TensorRT 11.2 deployment and need to control parsing in C++ or through the Python backend, including flags such as kNATIVE_INSTANCENORM. Do not adopt it if you want framework-independent inference, since every engine it produces is TensorRT and NVIDIA hardware only, and do not adopt it as a way to avoid upgrading TensorRT, because the branch tracks one TensorRT version at a time.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap onnx-tensorrt fills between an ONNX file and a TensorRT engine
An ONNX file is a protobuf graph. TensorRT does not consume that graph. It consumes a network definition built through its own API, and the engine it produces is tied to the GPU architecture and TensorRT version that built it. Something has to walk the ONNX nodes, check that each operator has a TensorRT equivalent, map attributes and initializers across, and report what it cannot handle. That something is onnx-tensorrt, packaged as libnvonnxparser.so with its C++ API declared in NvOnnxParser.h. The README describes the project in one line: it parses ONNX models for execution with TensorRT. The audience is narrow and practical. You are deploying to NVIDIA hardware, you already have a model exported to ONNX from PyTorch or another framework, and you need to know whether that specific graph will become a runnable engine. The parser is also the component that surfaces the answer when it will not, which is often the more valuable half of its job.
How the parser maps ONNX nodes onto TensorRT layers
The flow is serial rather than clever. You load an ONNX model, hand it to the parser bound to a TensorRT network and builder, and the parser walks the graph node by node. For each node it looks up an implementation, translates the ONNX attributes into the corresponding TensorRT layer parameters, and wires the layer into the network. Dynamic shapes and full dimensions are supported on this branch, which matters because exported models rarely have every dimension fixed at export time. The parser is also where per-operator behaviour is decided. The README's InstanceNormalization note is the clearest example: there are two implementations, a native TensorRT one and a plugin one, and the parser flag kNATIVE_INSTANCENORM selects between them. The default is native. Unsetting the flag switches to the plugin, which the README says may perform differently depending on parameters, and which cannot be used when building version-compatible or hardware-compatible engines. Attempting that combination raises an error rather than silently degrading. That constraint is the kind of thing you only learn by reading the flag documentation, and it directly shapes how portable the resulting engine can be.
Building the parser against a TensorRT 11.2 tree
The build is a plain CMake flow with two paths to a dependency. You need TensorRT 11.2 and the TensorRT 11.2 open source libraries; Protobuf 3.20.3 or newer is listed as optional. The README's commands are short:
cd onnx-tensorrt mkdir build && cd build cmake .. -DTENSORRT_ROOT=<path_to_trt> && make -j export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH
The LD_LIBRARY_PATH export is not decoration. Without it the freshly built library is not found at runtime, and the failure appears later as a load error rather than a build error. CUDA is a hard dependency and the build looks in /usr/local/cuda by default; override with -DCUDA_TOOLKIT_ROOT_DIR=<path_to_cuda_install> when the toolkit lives elsewhere. If you want protobuf-lite, append -DUSE_ONNX_LITE_PROTO=1 to the cmake invocation. For Windows or Docker builds the README points at the main TensorRT repository's build environment instructions instead of repeating them here. On the Python side, bindings are packaged in the shipped .whl files, TensorRT 11.2 pairs with ONNX 1.21.0 installed via python3 -m pip install onnx==1.21.0, and python3 setup.py install installs the backend.
Checking a model without writing parser code first
Before you build anything, the README names two officially supported tools for answering the only question that matters early on: does this ONNX model parse and build into a TensorRT engine. For C++ users there is trtexec, usually in <tensorrt_root_dir>/bin, invoked as trtexec --onnx=model.onnx, with trtexec -h listing the rest of the CLI options. For Python users there is Polygraphy, invoked as polygraphy run model.onnx --trt, with polygraphy run -h for options. Both are external to this repository and live in the TensorRT tree. That is a deliberate boundary: onnx-tensorrt supplies the parser, and the TensorRT project supplies the tools that exercise it. If you are evaluating the project, start with one of those two commands against your real model. A parse failure there is the same failure you would hit after writing integration code, minus the integration code. Once a model passes, the Python backend usage in the README is the shortest path to running it: load with onnx.load, call backend.prepare(model, device='CUDA:1'), then engine.run(input_data)[0] returns the output array.
Operator coverage is the real boundary, and it is documented separately
The README does not list supported operators inline. It points to docs/operators.md, the operator support matrix, and that file is where adoption decisions actually get made. This is the project's sharpest limitation and it is structural rather than a bug. TensorRT implements a fixed set of layers; ONNX defines a much larger operator set that grows with each ONNX release. Anything in your graph without a mapping is a parse failure, and the failure arrives at parse time, not at export time. Custom or recently standardized operators are the usual casualties. There is no fallback path described in the README for an unmapped node, and no plugin authoring guidance here either; the InstanceNormalization plugin is an implementation detail of the parser, not a general extension mechanism. So the honest framing is that onnx-tensorrt is a translation layer with a finite dictionary. Whether your model fits is a property of your model, not of the parser's quality, and docs/operators.md plus a single trtexec run will tell you more in five minutes than any amount of reading.
Where onnx-tensorrt stops and ONNX Runtime begins
The obvious alternative is ONNX Runtime, and the difference is not performance tuning, it is what each one produces. ONNX Runtime executes the ONNX graph itself across a range of execution providers and hardware. onnx-tensorrt does not execute ONNX. It converts, and the artifact is a TensorRT engine bound to a specific TensorRT version and GPU architecture. That distinction drives everything downstream. With ONNX Runtime you keep the model as the deployable unit and can move it between machines. With onnx-tensorrt you keep an engine, and the README's own note about version-compatible and hardware-compatible engines, and which parser flags are incompatible with them, shows how much that binding matters in practice. The other difference is tooling ownership. ONNX Runtime is the runtime; here the runtime is TensorRT, and the diagnostic tools are trtexec and Polygraphy from the TensorRT repository. If your constraint is NVIDIA deployment and you want the engine-building control that TensorRT exposes, including parser flags, this project is the intended path. If your constraint is portability across hardware, it is the wrong tool regardless of how well your operators are covered.
Version lock, release cadence and the Apache-2.0 terms
The branch tracks TensorRT 11.2, and the README states plainly that previous TensorRT versions live on their respective branches. The release list shows what that means operationally: release/11.0-GA, release/11.1-GA and release/11.2-GA, roughly two months apart in the supplied dates. Each is described as a parser update tied to a TensorRT GA. So the upgrade unit is not a patch bump, it is a coordinated move of TensorRT, the parser branch, and the ONNX version the README pairs with it (1.21.0 for TensorRT 11.2). Engines built before the move are not automatically usable after it. Budget for that as a recurring cost rather than a one-time integration. On licensing, the repository is Apache-2.0, with SPDX headers in the source, including the README itself. That covers the parser code. It does not cover TensorRT, which is a separate NVIDIA download under its own terms, and it does not cover the ONNX Model Zoo models the README points to. Check those separately; this is a description of what the files say, not legal advice.
Editorial conclusion
Adopt onnx-tensorrt if you are already pinned to a TensorRT 11.2 deployment and need to control parsing in C++ or through the Python backend, including flags such as kNATIVE_INSTANCENORM. Do not adopt it if you want framework-independent inference, since every engine it produces is TensorRT and NVIDIA hardware only, and do not adopt it as a way to avoid upgrading TensorRT, because the branch tracks one TensorRT version at a time. Verify first that every operator in your graph appears in docs/operators.md for your TensorRT release, then run trtexec --onnx=model.onnx or polygraphy run model.onnx --trt before writing any integration code.
Community notes