pnnx and the problem of getting a PyTorch model out of PyTorch

The repository is a signpost, the implementation lives in ncnn

The first thing the project guide tells you is that the current implementation is at a different address, inside the Tencent ncnn repository under tools/pnnx.

That single line reframes what pnnx/pnnx is. It is not the tool, it is the specification and the announcement point. The name expands to PyTorch Neural Network eXchange, and the project describes itself as an open standard for PyTorch model interoperability that provides an open model format for PyTorch, defining the computation graph as well as high level operators in a way that strictly matches PyTorch.

The distinction matters for anyone reading the repository. The top-level tree here contains a project description file, images/, a setup.py, a .github/ directory, and then a long list of patch files with names like onnxruntime-1.30.0-monolithic-static-library.patch, protobuf-33.6-align-abseil.patch and pytorch-v2.14.0-fix-mobile-build.patch.

Those patches are the clearest signal of what this repository is for. They are fixes applied to third-party build dependencies, versioned to specific releases, and they are the part of the work that has to live somewhere shared. The tool itself, the actual graph conversion, lives in ncnn.

So if you are evaluating pnnx, you are evaluating a format and a set of build fixes, with the implementation hosted elsewhere.

Four things it does with a model, and they are not the same thing

The project guide presents pnnx as four capabilities, and the differences between them are the point.

Optimizing a torch model is the first. Reduce extension package dependency is the second. Convert torchscript or onnx back to python is the third, which is a genuinely different direction of travel from the others. Export to portable pnnx format, onnx-zero or ncnn is the fourth.

The conversion graph in the guide makes the flow concrete. A torch model can be traced to a torchscript file with torch.jit.trace and saved, or exported to onnx with torch.onnx.export. Other frameworks, listed as caffe, mxnet, keras, tensorflow and paddlepaddle among others, export to onnx first. From there, pnnx.export takes a torch model directly, and pnnx.convert handles a torchscript file, an onnx file or a tnn file, a tnnproto and tnnmodel pair.

And the outputs are equally varied. An optimized torch model is a resnet18_pnnx.Model() object. An ncnn model is a resnet18.ncnn.param and resnet18.ncnn.bin pair. An onnx-zero model is a resnet18.pnnx.onnx file. A tnn file is the ncnn-adjacent format.

The diagram also shows where pnnx sits internally, with the optimized model, the ncnn model and the onnx-zero model grouped together, which suggests they are alternative renderings of the same internal graph rather than separate pipelines.

Installing it: pip, a portable binary, or build inside ncnn

Three install paths are given, and the choice depends on how much of the surrounding stack you already have.

The recommended Python route covers Windows, Linux and macOS on 64bit with Python 3.7 or later:

pip3 install pnnx

The portable binary package is described as recommended if you hate Python. It covers the same three platforms, needs glibc 2.17 or newer on Linux, and is downloaded from the releases page and extracted. The guide makes a point about this one: the package includes all the binaries required, and it is portable, so no CUDA or PyTorch runtime environment is needed.

That last sentence is the differentiator. Anyone who has fought a PyTorch and CUDA install on a CI runner will recognise why a self-contained binary that needs neither is worth having, even if it means a second copy of the tool rather than reusing the environment's Python.

Building from source is the third path and it is not a build of this repository. The steps are to install pytorch, optionally install torchvision for the pnnx torchvision operator support, optionally install protobuf for pnnx onnx-zero support, clone the ncnn repository, and build pnnx in ncnn/tools/pnnx with cmake:

git clone https://github.com/Tencent/ncnn.git
mkdir ncnn/tools/pnnx/build
cd ncnn/tools/pnnx/build
cmake -DCMAKE_INSTALL_PREFIX=install -DTorch_INSTALL_DIR=<your libtorch install dir> -DTorchVision_INSTALL_DIR=<your torchvision install dir> ..
cmake --build . --config Release -j 4
cmake --build . --config Release --target install

The two Torch directories are parameters, so you point the build at an existing libtorch and torchvision install. The guide says you will probably want to refer to the release workflow in the .github directory for detailed steps, which is where the version-specific patching described earlier is applied.

The Python API is one export call and a usable module

Using pnnx from Python is a three step flow, and the middle step is the part that makes the tool worth the setup.

First you export and optimize the model:

import torch
import torchvision.models as models
import pnnx

model = models.resnet18(pretrained=True)

x = torch.rand(1, 3, 224, 224)

opt_model = pnnx.export(model, "resnet18.pt", x)

# use tuple for model with multiple inputs
# opt_model = pnnx.export(model, "resnet18.pt", (x, y, z))

The export call takes the model, a filename and an example input tensor, and the example input matters because the graph rewrite works on a concrete shape. The comment about using a tuple for a model with multiple inputs says the rest.

Second, you use the optimized module just like the normal one:

result = opt_model(x)

That is the whole appeal. The optimized module is a drop-in replacement in Python, so you can keep training or evaluating in PyTorch while the converted artifacts serve another runtime.

Third, you pick the artifacts for whichever target you need: resnet18_pnnx.py for the pnnx-optimized torch model, or resnet18.ncnn.param and resnet18.ncnn.bin for ncnn inference.

Note the direction of the tool. pnnx.export is an optimization pass, and the onnx-zero output is described elsewhere in the guide as a model that can be brought back to Python, which is the convert-backwards capability in practice.

The command line path needs the input shape spelled out

The command line route starts from a torchscript or onnx file rather than from a live model. You export first, in Python:

import torch
import torchvision.models as models

net = models.resnet18(pretrained=True)
net = net.eval()

x = torch.rand(1, 3, 224, 224)

# You could try disabling checking when tracing raises error
# mod = torch.jit.trace(net, x, check_trace=False)
mod = torch.jit.trace(net, x)

mod.save("resnet18.pt")

# You could also try exporting to the good-old onnx
torch.onnx.export(net, x, 'resnet18.onnx')

The commented check_trace=False line is worth noting, because tracing a model with data-dependent control flow is the classic failure mode and the tool offers an escape hatch for it.

Then pnnx converts the file, and the input shape has to be given on the command line:

./pnnx resnet18.pt inputshape=[1,3,224,224]
./pnnx resnet18.onnx inputshape=[1,3,224,224]
./pnnx resnet18.tnnproto inputshape=[1,3,224,224]

There are three variations on that syntax, and each solves a specific problem. macOS zsh users may need double quotes to prevent ambiguity, as the guide says, and the quoted form is shown separately. A model with multiple inputs uses a list, written as inputshape=[1,3,224,224],[1,32]. And a model with a non-fp32 input data type takes a type suffix, written as inputshape=[1,3,224,224]f32,[1,32]i64.

The type suffix is the one to remember. Without it the shape is ambiguous about the element type, and the same bracketed syntax carrying an f32 or i64 marker is how you express it.

Seven files come out, and Netron can open the graph

The guide says that normally you will get seven files, and then it lists the outputs in a table that is cut off partway through the descriptions.

What is visible is the naming scheme and the division of labour. resnet18.pnnx.param is the PNNX graph definition. resnet18.pnnx.bin is the PNNX model weight. resnet18.pnnx.py is the PyTorch script for inference, described as the py on the truncated line.

Splitting the graph from the weights is the format's most transferable property. A .param file plus a .bin file is a pair of opaque blobs that a runtime can load, while the .py file is readable Python that shows you what the graph actually became after the rewrite. That matters when the conversion does something you did not expect, because you can read the Python rather than reverse-engineering a binary.

For inspection, the guide points at Netron. Open https://netron.app/ in a browser and drag resnet18.pnnx.param or resnet18.pnnx.onnx into it. Both the graph definition and the onnx-zero output are viewable, which makes it possible to check the conversion before shipping it to a target runtime.

The guide also has a section on comparing inference results, which is the step that catches a bad conversion, and the onnxruntime and pytorch patch files at the repository root relate to keeping those comparisons reproducible against specific dependency versions.

The patch files are the maintenance story

The most unusual thing about this repository is the set of dependency patches, and they deserve a section of their own because they explain the project's real work.

There are four onnxruntime patches, all versioned to onnxruntime-1.30.0, covering a less-mlas-features variant, a monolithic static library variant, and a use-clog change. There is a protobuf patch, protobuf-33.6-align-abseil.patch, aligning a build dependency with abseil. There are seven pytorch patches versioned to pytorch-v2.14.0, covering an eigen build fix, a link-local sleef fix, a mobile build fix, a stable ABI fix, two no-arm64 sleef fixes covering the non-SVE and SVE variants, a no-link-system-lib fix, a no-mimalloc fix, a revert-nativert-api fix, a set-python-executable fix. And there are three vision patches versioned to vision-0.29.0.

Each of these is a build fix against a specific upstream version, and each will need reapplying when that version moves. That is a maintenance obligation that does not scale with your own code and does not show up in your dependency manifest.

It also tells you something about the environment pnnx has to live in. A fix to a monolithic static library build of onnxruntime, a fix to align protobuf with abseil, fixes to sleef for arm64, a revert of an API in a native runtime, and a fix to a mobile build of PyTorch are all problems of assembling a cross-platform inference stack from several heavy C++ dependencies, and they are the kind of problem that appears when the same model has to run on a phone, a desktop and a server.

The repository also carries an .github/ directory with the release workflow the build instructions point to, which is where those patches are applied.

Editorial conclusion

pnnx suits a team that needs to ship one PyTorch model to runtimes that cannot load PyTorch, especially ncnn on mobile, and that is willing to accept a conversion step with a graph-level output rather than a container format. It does not suit someone who wants a stable, centrally published package, because the implementation now lives in the ncnn repository and this one carries no releases of its own in the usual sense, and it does not suit a project that must keep bit-identical numerics across backends, since the tool's job is to rewrite the graph. Before you adopt it, read the note that the current implementation is in the ncnn tools/pnnx directory, check the version of PyTorch you are on against the release workflow it points to, and note that the last push was on 2026-10-20 with the newest listed release dated 2026-07-04.

Frequently asked questions

Official sources

1. Issues
2. pnnx/pnnx on GitHub
3. README
4. Releases