PoinTr: A Geometry-Aware Transformer for Point Cloud Completion

What PoinTr completes, and for whom

A partial scan of a chair, a car or a plane is missing the surfaces the sensor never saw. PoinTr takes that incomplete cloud and predicts the missing geometry, producing a dense, complete point set. The repository is the PyTorch implementation behind the ICCV 2021 oral paper, and the README frames the contribution as representing the cloud as a set of unordered groups of points with position embeddings, turning it into a sequence of point proxies that a transformer encoder-decoder can consume. That framing is the whole idea: completion is treated as sequence generation over learned proxies rather than as voxel or folding-based reconstruction. The intended audience is researchers and engineers who already have point cloud data and a GPU, and who want a published baseline to reproduce, fine-tune or compare against. The README also points to ShapeNet-55/34, described as two more challenging benchmarks with more diverse incomplete point clouds, which the authors position as closer to real-world capture than the earlier PCN split. If your goal is a production completion service rather than an experiment, note that the repository ships inference and evaluation scripts, not a serving layer.

The encoder-decoder path from partial cloud to dense output

The architecture visible in the material is a transformer encoder-decoder over point proxies. A partial cloud is grouped into unordered point sets, each group carries a position embedding, and the resulting sequence of proxies is what the transformer attends over. The decoder generates the completed point set. Two design consequences follow from this representation. First, the model never sees a fixed grid, so it is not tied to a voxel resolution. Second, the grouping step is where locality is imposed, since attention itself is permutation-invariant and carries no notion of which points are neighbours. The repository also contains a second-generation variant, AdaPoinTr, described in the README as PoinTr plus Adaptive Denoising Queries and reported as accepted by T-PAMI, with its own pretrained checkpoints listed for ShapeNet-55, ShapeNet-34, Projected_ShapeNet-55/34 and PCN. SnowFlakeNet is implemented in the same tree as of June 2022. The practical point is that this is a family of models sharing one training and evaluation harness, not a single frozen network. Which config file you pass to the tools determines which of them you actually run.

Installing the CUDA extensions is the real gate

The Python dependencies are ordinary: the README lists torchvision, timm, open3d and tensorboardX, installed with pip install -r requirements.txt, on top of PyTorch >= 1.7.0, python >= 3.7, CUDA >= 9.0 and GCC >= 4.9. The part that decides whether you get anywhere is the compiled code. Chamfer Distance is built with bash install.sh, and the README links a known installation bug to issue #6 rather than fixing it in the script. PointNet++ is pulled from a git URL with a subdirectory fragment, and GPU kNN is installed from a prebuilt wheel hosted on a GitHub release. If an extension still fails to import, the README's documented fallback is explicit: cd into extensions/<Module>, for example extensions/gridding, and run python setup.py install. That fallback exists because ModuleNotFoundError for gridding is common enough to warrant its own note. Treat the extension build as the first milestone, not an afterthought. A machine that cannot compile these will not run the evaluation scripts, regardless of how clean the Python environment looks.

Running inference and evaluation with the shipped scripts

Inference is a single script with a config and a checkpoint as positional arguments. The README gives this example, which infers every sample under demo/ and writes results to inference_result/ while saving visualisations:

python tools/inference.py cfgs/PCN_models/AdaPoinTr.yaml ckpts/AdaPoinTr_PCN.pth --pc_root demo/ --save_vis_img --out_pc_root inference_result/

The optional flags matter: --pc_root points at a directory, --pc at a single file, --save_vis_img writes visual output, and --out_pc_root chooses where completed clouds land. Evaluation goes through a shell wrapper instead: bash ./scripts/test.sh <GPU_IDS> --ckpts <path> --config <config> --exp_name <name>, with an optional --mode <easy/median/hard> that selects the difficulty split. The README's own example is bash ./scripts/test.sh 0 --ckpts ./pretrained/PoinTr_PCN.pth, which runs on GPU 0 against the PCN checkpoint. Note the asymmetry: inference takes the config as a positional argument, evaluation takes it behind a --config flag. Getting that wrong is a silent failure mode, because the script will look for a checkpoint path that does not exist rather than complain about argument order.

Benchmark numbers and what they do not tell you

The README publishes Chamfer Distance for each checkpoint: ShapeNet-55 at 1.09e-3, ShapeNet-34 at 2.05e-3, PCN at 8.38e-3, and PCN_new at 7.26e-3, the last reflecting a bug fix in datasets/PCNDataset.py that the changelog records as moving PCN from 8.38 to 7.26. KITTI is reported with MMD = 5.04e-4 instead of CD, because the ground truth there is sparse and CD is a poor fit. The AdaPoinTr table lists ShapeNet-55 at 0.81e-3, ShapeNet-34 at 1.23e-3, Projected_ShapeNet-55 at 9.58e-3, Projected_ShapeNet-34 at 9.12e-3 and PCN at 6.53e-3. Read those numbers with care. They are self-reported, they are computed by the same Chamfer Distance extension you have to compile yourself, and the PCN row changed once already after a dataset bug. The gap between ShapeNet-34 and ShapeNet-55 is also not a quality difference in the model; it reflects how the categories are split. Comparing a number from this table against a number from another paper without checking the split and the metric is the easiest way to draw a wrong conclusion.

Where PoinTr is the wrong choice

This is a research repository, and several constraints follow from that. There are no packaged releases in the material provided, so there is no versioned artifact to pin and no upgrade path beyond tracking the master branch. The dependency chain includes a git URL and a wheel hosted on a third-party GitHub release page, which means your build depends on external hosts staying reachable. The environment floor is old in places: PyTorch >= 1.7 and CUDA >= 9.0 are stated as minimums, and the extension code is compiled against whatever CUDA toolkit you have, so a mismatch between the toolkit used to build and the runtime is a plausible source of failures that the README does not address. There is no inference server, no batching API and no documented latency or memory figures, so capacity planning for a deployment is not something this material supports. And if your clouds are large outdoor LiDAR scenes rather than object-centric shapes, the benchmarks here (ShapeNet variants, PCN, KITTI) do not cover that regime; the KITTI checkpoint exists, but it is one entry among several and the README gives no guidance on domain transfer.

How PoinTr differs from folding-based completion

The obvious comparison is with folding-based methods such as SnowFlakeNet, which the README notes was implemented in this repository in June 2022. The two take opposite routes to the same output. Folding approaches predict a deformation from a fixed 2D grid into 3D space, so the number of output points is determined by the grid resolution and the network learns a warp. PoinTr instead keeps the cloud as unordered point groups, embeds them as proxies, and lets a transformer decoder generate the missing points directly. The difference shows up in what each method is good at. A folding decoder produces a regular, evenly distributed output by construction, which suits surfaces that are roughly manifold. A proxy-and-attention decoder has more freedom to place points where the evidence suggests geometry, which is why the paper's framing is about diversity of incomplete shapes rather than uniformity of output. The cost is that PoinTr's output density is a learned property rather than a grid parameter, and the grouping step adds a hyperparameter that folding methods do not have. Neither is strictly better; they fail differently.

Licence, maintenance and the cost of staying current

The repository is MIT licensed, which permits commercial use and modification provided the copyright notice and permission notice are retained. That is a permissive starting point, but it covers the code in this repository only. The build pulls in PointNet++ from erikwijmans/Pointnet2_PyTorch and a kNN wheel from a separate GitHub release, each with its own terms, and the pretrained checkpoints are distributed through Tsinghua Cloud, Google Drive and BaiDuYun links rather than a package registry. Anyone shipping a product on top of these weights should check the terms attached to the datasets they were trained on; the README points to DATASET.md but does not restate dataset licences. On maintenance: the last push recorded is 2026-06-24, and the changelog's most recent entry is the September 2023 AdaPoinTr acceptance. The project is alive but not fast-moving, and the absence of tagged releases means there is no changelog to diff between versions. Budget for reading commits rather than release notes when you upgrade.

Editorial conclusion

Adopt PoinTr if you need a published, MIT-licensed baseline for point cloud completion and you are willing to compile CUDA extensions before any model code runs. Do not adopt it if you need a pip-installable library with a stable API, or if you cannot match PyTorch >= 1.7, CUDA >= 9.0 and GCC >= 4.9 on the machine that will train or evaluate. Before you commit, verify that install.sh completes and that the gridding, Chamfer Distance and kNN modules import; the README's own fallback is to run python setup.py install from inside extensions/<Module>. If that step fails, nothing downstream in this repository will run.

Official sources

1. Issues
2. License: MIT
3. README
4. yuxumin/PoinTr on GitHub