tensorrtx: hand building TensorRT networks one layer at a time
Implementation of popular deep learning networks with TensorRT network definition API
At a glance
- What is it?
- A C++ collection of deep learning architectures assembled through the TensorRT network definition API rather than loaded from an ONNX file, useful as a working reference and less useful as a shortcut.
- Who is it for?
- Tensorrtx is a reference implementation rather than a deployment tool. It earns its place if you are learning how a specific architecture maps onto TensorRT layers, or if you need to edit a graph at the C++ level and fold preprocessing into the engine.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 55 days ago.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 11, 2026, and from our analysis. They are not legal advice.
Editorial analysis
Building networks layer by layer instead of parsing a model file
The repository's opening question is the one worth sitting with: why not use a parser, the README asks, listing the ONNX parser, the UFF parser and the caffe parser as the obvious alternatives, and instead use the more complicated TensorRT network definition API to build a network from scratch. Three answers are given. The network is flexible, since you can add or delete a layer or an input or output tensor, replace a layer, merge layers, and fold preprocessing and postprocessing into the network itself. It is debuggable, because you construct it incrementally and can inspect intermediate layer results. And it is educational, since you learn the structure instead of treating the graph as a black box.
Folding preprocessing into the engine is the capability that matters most in production. A graph that begins with a decode, a resize and a normalisation, and ends with a threshold and an NMS pass, removes a whole class of host to device round trips. You cannot get that from a parser alone without writing plugin layers anyway, and once you are writing plugins you are back in the definition API.
The trade is real though. Every tensor, shape and layer is a line of C++ you have to get right, and a wrong one shows up as a build failure or a silent numerical mismatch rather than a clear error. This is a repository of roughly fifty such implementations, not a library that abstracts the API away.
The four step path from pytorch weights to an engine
The README describes the workflow in four steps, and the sequence is what makes the project reproducible. Step one is to obtain a trained model, from pytorch, mxnet or tensorflow, with some pytorch models mirrored in a companion repository called pytorchx and the rest taken from established open source projects. Step two is to export the weights to a plain text file, which the project calls a .wts file and documents in its own format section. Step three loads those weights into TensorRT, defines the network and builds an engine. Step four loads the engine and runs inference.
The .wts file is the pivot of the whole design, and it is worth understanding why it is plain text rather than a binary blob. A human can open it, and a conversion script in pytorchx can be compared against the loader that consumes it. When a network produces wrong output, the first question is whether the weights arrived intact, and a readable intermediate file makes that answerable by looking.
The workflow also explains why there is no Python in the tree. Because the graph is defined in code rather than read from a file, every architecture is a self contained C++ program with its own CMakeLists.txt, and getting the layer order right is a deliberate act rather than a side effect of the exporter.
One subproject at a time, or the whole tree with Ninja
Builds come in two documented shapes, and picking the single subproject form is the sensible default even though the README presents both as equals. Every model directory carries its own CMakeLists.txt, so you configure and build just the one you want:
## enter the subfolder
cd tensorrtx/xxx
## configure & build
cmake -S . -B build
make -C buildThe alternative configures the top level CMakeLists.txt and builds everything the file has enabled:
## enter the top of this project
cd tensorrtx
## configure & build
# you may use "Ninja" rather than "make" to significantly boost the build speed
cmake -G Ninja -S . -B build
ninja -C buildWith the top level build you have to edit that file by hand, uncommenting the projects you want and leaving out the rest, because the README notes you cannot build the targets grouped under a version condition if your TensorRT is older. That is exactly the maintenance burden the single subproject route avoids. The README also attaches a warning to this part, saying it is still under development and that most subprojects are not adapted yet, which is a fair signal to treat the whole tree build as a convenience rather than a supported path.
Serializing and deserializing the engine with two flags
The generated executable takes a small set of flags, and the pair worth knowing is the one that separates building a plan file from running it. The minus s flag serializes the model to a plan file, and the minus d flag deserializes that plan and runs inference:
# serialize model to plan file i.e. 'xxx.engine'
build/xxx -s
# deserialize plan file and run inference
build/xxx -dSplitting the two is what makes the project usable in a service. Engine construction is the slow part, it is the step that touches your GPU for minutes, and doing it once at deploy time means the process that serves traffic only loads a file. The README also points at two optional checks: comparing the output against the pytorchx reference, and customising the project.
Beyond that, the tutorials directory carries the pieces that a working example cannot teach on its own. There is a getting started guide that walks through lenet5 end to end, a migration guide, a page on implementing multi GPU processing using YOLOv4 as the example, a page for checking whether your GPU supports FP16 and INT8, a Windows compile and run guide, a Triton Inference Server deployment example, and a Chinese-language step by step walkthrough taking hrnet from pytorch to TensorRT. Each model directory can also ship its own README with specifics.
Fifty two model directories, and a changelog kept by named contributors
The scale of the collection is the repository's main asset. Of the fifty five top level directories, three are infrastructure (.github, docker and tutorials) and fifty two are architectures. They span classification from lenet and alexnet through resnet, densenet, regnet, repvgg and vit; detection through yolov1, yolov3, yolov4, yolov5, yolov7, yolo10, yolo11, yolo12, yolo13, yolop, rcnn, detr, refinedet and centernet; text and sequence work through crnn, arcface, lprnet and ufld; face work through retinaface and retinafaceAntiCov; and restoration through real-esrgan. A qwen3 directory for Qwen3-0.6B puts a language model in the same frame.
The News section is maintained by crediting contributors by handle rather than by version number, and it reads as a genuine community log. Qwen3-0.6B support arrived on 11 Jul 2026 from mpj1234. On 24 Jun 2026 the default branch switched to trt10. Earlier entries add PP-OCRv5 and related models, Vision Transformer, YOLOv13, YOLO11, YOLOv8-obb, GhostNet and real-esrgan. One entry from January 2026 refactors older computer vision models to support TensorRT SDK versions seven through ten, which is the kind of sweeping change that tends to break something.
That contributor structure matters for expectations. There is no maintainer roadmap and no release process, so capability arrives when somebody needs a model. The repository is not archived and the last push was on 2026-08-18, which means the branch you clone is current work, not a maintenance snapshot.
Version guidance that disagrees with itself
Two inconsistencies in the documentation are worth knowing about before you commit an afternoon to a build. Neither is fatal, but both cost time if you discover them halfway through.
The first is the TensorRT version. The Test Environment section states a single requirement, TensorRT 10.x or later. The Note section immediately below it says something different, that for historical reasons some models are limited to a specific TensorRT version and that you should check the README or code for the model you want, and then adds that TensorRT 8.x currently has better compatibility with most of the features supported. Those two statements pull in opposite directions, and the resolution is the per model guidance: the top level number is where the branch is heading, the 8.x note describes where coverage is widest today, and the README of the individual model directory is the authority for the one you are building. The January 2026 refactor to support versions 7 through 10 is consistent with coverage spread across three majors rather than concentrated in one.
The second is naming drift. The repository metadata names main as the default branch, and the June 2026 news entry says the default branch switched to trt10 as main. Several links inside the News list still point at master in the repository URL, including the Lenet Tripy example and the YOLO11-obb entry. The repository topics list yolov8 and yolov9, and neither has a directory, while yolo10, yolo12 and yolo13 do; the same list also mentions a YOLO11 written in Tripy under a yolo11_tripy directory that the tree does not contain. Topics and changelog entries accumulate more slowly than directories get added, so treat the directory listing as the real inventory. There are also no tagged releases at all, which means there is no version to pin and no upgrade notes beyond the News entries.
Editorial conclusion
Tensorrtx is a reference implementation rather than a deployment tool. It earns its place if you are learning how a specific architecture maps onto TensorRT layers, or if you need to edit a graph at the C++ level and fold preprocessing into the engine. For shipping inference, a framework exporter or NVIDIA's Tripy gets you to a plan file with far less code. Concretely: open the README of the one model directory you care about, confirm which TensorRT major version it targets, and treat that directory's build rather than the repository root as the unit of work. There are no tagged releases, so the branch you check out is the version you get.
Frequently asked questions
What does tensorrtx actually do?
It implements popular deep learning networks through the TensorRT network definition API, meaning the graph is assembled in C++ code rather than loaded from an ONNX, UFF or caffe file. Roughly fifty two model directories cover classification, detection, text recognition, face detection and image restoration, each with its own build target and README.
What is a .wts file and why does tensorrtx need one?
A .wts file is a plain text export of trained weights produced from a framework such as pytorch, mxnet or tensorflow. Because the network definition is hand written rather than parsed, weights cannot travel inside a model file, so they are loaded separately at build time. The plain text format also means the file can be inspected and compared when a network produces wrong output.
Which TensorRT version should I use with tensorrtx?
The Test Environment section states TensorRT 10.x or later, while the note below it says 8.x currently has better compatibility and supports most features. Both are in the README as written. The workable answer is to read the README inside the specific model directory you need, since older architectures are pinned to particular versions and newer ones follow the trt10 default branch.
How do you build one model instead of the entire repository?
Each model folder has its own CMakeLists.txt, so you enter that folder and run cmake with the source and build directory, then build the resulting directory. The README also documents a top level build that compiles many models at once, but it requires hand editing the root CMakeLists.txt and the project warns that part is still under development.
Is tensorrtx a faster route to inference than using an ONNX parser?
Not as a matter of the resulting engine, since both paths can produce a comparable plan file. The difference is engineering effort: a parser gets you from a trained model to an engine with far less code, while hand definition costs more work and returns the ability to modify layers, inspect intermediates and move preprocessing into the graph.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/wang-xinyu-tensorrtx)