TensorRT OSS: the open source half of NVIDIA's inference stack, after the 11.x API break
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
At a glance
- What is it?
- This repository ships the open source components of TensorRT (plugins, ONNX parser, samples) rather than the inference engine itself, and TensorRT 11.0 removed weakly-typed networks, implicit quantization and IPluginV2. Here is what that means for anyone deciding whether to build against it.
- Who is it for?
- Adopt TensorRT OSS if you are already pinned to an NVIDIA GPU deployment and your code targets strongly typed networks, explicit quantization and IPluginV3, because the 11.x removals mean anything written against the 10.x plugin or quantization APIs will not carry over. Do not adopt it as a portable inference layer: the engine is a separate GA download, the OSS repo is a subset, and the build expects a specific CUDA and TensorRT version pairing.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 5 days ago.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What the repository actually contains, and what it does not
The README is explicit that this repository holds the Open Source Software components of TensorRT, described as a subset of the TensorRT General Availability release with some extensions and bug fixes. The named contents are the sources for TensorRT plugins and the ONNX parser, plus sample applications. The inference engine that actually executes a plan on the GPU is not in this repository. You obtain it separately from the NVIDIA Developer Zone, and the README pins the matching version: TensorRT v11.2.1.2.
That split defines the audience. If you are shipping a model to an NVIDIA GPU and you need a parser for an ONNX graph, a set of standard plugins, or a worked sample to copy from, this is the code you read. If you want to compile and run inference without touching a build system, the README points at a different path entirely: pip install tensorrt, and then skip the Build section. The repository is for people who need to modify or rebuild the components, not for people who just want the runtime.
The 11.x break: three APIs removed, not deprecated
The announcement section lists the removals that make TensorRT 11.x a migration rather than an upgrade. Weakly-typed networks and their related APIs are gone, replaced by strongly typed networks. Implicit quantization is gone, replaced by explicit quantization. IPluginV2 and its related APIs are gone, replaced by IPluginV3. The TREX tool is gone, replaced by Nsight Deep Learning Designer.
Each of those is a rewrite, not a flag flip. A plugin author who implemented IPluginV2 has to port to IPluginV3; the README links a migration page titled migrating v2 plugins to IPluginV3. A team relying on implicit quantization has to move to explicit quantization, which means the quantization information has to be present in the model rather than inferred at build time. Weakly typed networks required the builder to infer tensor types; strongly typed networks require them to be declared. The practical effect is that upgrading from 10.x is a code change in your model conversion path, and the README treats it as such by publishing migration guides rather than a compatibility shim.
Packaging changed too. Python bindings for Python 3.9 and older are removed, and the RPM packages for RHEL/Rocky Linux 8 and 9 now depend on Python 3.12. That is a deployment constraint on older enterprise distributions, not just a developer inconvenience.
Getting a build: the prerequisites that actually gate you
The build section is unusually specific, which is useful because the version pairings are where builds fail. You need the TensorRT GA build at v11.2.1.2. CUDA is recommended at 13.3.0 or 12.9.0. cuDNN 8.9 is listed as optional. On the toolchain side: GNU make 4.1 or newer, CMake 3.31 or newer, Python between 3.10 and 3.14.x, pip 19.0 or newer, plus git, pkg-config and wget.
The clone sequence from the README is:
git clone -b main https://github.com/nvidia/TensorRT TensorRT cd TensorRT git submodule update --init --recursive
The recursive submodule step matters because the README notes that onnx-tensorrt, cub and protobuf are downloaded along with TensorRT OSS and do not need to be installed separately. If you skip submodules, you will be missing the ONNX parser sources that are one of the two main reasons to be in this repository at all.
If you are not using the TensorRT OSS build container, you must download and extract the GA build yourself and point the build at it; inside the container the libraries are preinstalled under /usr/lib/x86_64-linux-gnu and that step can be skipped. NCCL is only needed when building with multi-device support via -DTRT_BUILD_ENABLE_MULTIDEVICE=ON, for the sampleDistCollective sample, and the README constrains it to NCCL 2.19 or newer and below 3.0. Containerized builds need Docker 19.03 or newer plus the NVIDIA Container Toolkit. The demo and test layer pulls in onnx, onnxruntime, tensorflow-gpu 2.5.1 or newer, Pillow 9.0.1 or newer, pycuda below 2021.1, numpy and pytest.
Where the documentation is thin and you are on your own
The README routes readers to a set of separate documents rather than explaining behaviour inline: an Import Workflows Guide covering the ONNX, Torch-TensorRT, HuggingFace/Optimum and Network Definition API paths, and a Supported Models matrix broken down by import path across LLM, encoder-NLP, vision, audio, diffusion and multimodal categories. It also links a changelog and a roadmap PDF for Q3 2026.
What the README does not give you is any statement about accuracy, latency or throughput for a given model. There is no benchmark table in the material, and the repository description does not claim one. If your adoption decision depends on a number, this repository will not supply it; you would have to produce that number on your own hardware, and the README's own framing (a subset of GA with extensions and bug fixes) suggests the OSS tree is not the artifact NVIDIA benchmarks.
The agentic coding skills directory, .agents/skills, is listed as containing skills related to TensorRT usage and benchmarking, with installation deferred to whatever coding agent you use. That is a thin description. Whether those skills are maintained at the same cadence as the parser and plugins is not something the supplied material answers.
The wrong tool for portable or non-NVIDIA inference
TensorRT is bound to NVIDIA GPUs by construction, and the repository description says so directly. The failure mode is not subtle: if your deployment target includes CPUs, AMD GPUs, Apple silicon or any accelerator that is not NVIDIA, this is not a candidate, and no amount of ONNX in the middle changes that, because the ONNX parser here exists to feed the TensorRT builder, not to run the graph.
ONNX Runtime is the obvious alternative and the difference in approach is structural. ONNX Runtime is a runtime that consumes the ONNX format directly and executes it across a set of execution providers, with TensorRT available as one of them. TensorRT is an ahead-of-time compiler: you build an engine for a specific GPU and a specific TensorRT version, and you deploy that engine. The ONNX Runtime path keeps the graph as the deployable artifact and picks the backend at runtime. That costs you the graph-level optimizations a dedicated builder can apply, and it buys you portability and a deployment artifact that does not need rebuilding when the target GPU changes. The README's own prerequisite list shows the cost of the TensorRT path: a pinned GA build, a pinned CUDA pairing, and a rebuild when those move.
One more boundary worth stating plainly. The repository is Apache-2.0, and the README carries the standard Apache 2.0 badge, but the GA build you download from the Developer Zone is a separate artifact with its own terms. The OSS licence here does not describe the whole stack, and nothing in the supplied material reconciles the two. That is a question for your own legal review, not something this repository answers.
Upgrade cost, and who should stay on 10.x
The release cadence visible in the material is fast: v11.0 on 2026-06-02, v11.1 on 2026-06-24, v11.2 on 2026-08-04, with the last push to main on 2026-08-25. Three minor releases in roughly two months, and the repository tracks main against a GA build that must match, here 11.2.1.2. The maintenance cost is therefore not in patching the OSS tree; it is in keeping your plugin code, your quantization path and your CUDA and TensorRT version pins aligned with a release train that moves on that schedule.
If you are on 10.x and your code implements IPluginV2 or depends on implicit quantization, the 11.x line is not a drop-in. The migration guides exist, but they describe a port. Staying on 10.x is a defensible position while that port is scoped, and the material does not state an end-of-support date for 10.x, so that window is not something the README quantifies.
For new work targeting an NVIDIA GPU, the calculus is different: there is no legacy to port, strongly typed networks and explicit quantization are the only options in 11.x, and the pip install tensorrt path lets you evaluate the runtime before you ever build the OSS components. The OSS repository becomes relevant at the point you need to change the parser or a plugin, or read a sample to understand the import path.
Editorial conclusion
Adopt TensorRT OSS if you are already pinned to an NVIDIA GPU deployment and your code targets strongly typed networks, explicit quantization and IPluginV3, because the 11.x removals mean anything written against the 10.x plugin or quantization APIs will not carry over. Do not adopt it as a portable inference layer: the engine is a separate GA download, the OSS repo is a subset, and the build expects a specific CUDA and TensorRT version pairing. Before committing, verify three things against your own tree: whether any of your plugins still implement IPluginV2, whether your quantization path depends on implicit quantization, and whether your Python runtime is 3.10 or newer, since bindings for 3.9 and older were removed and the RHEL/Rocky Linux RPMs now depend on Python 3.12.
Community notes