NKSR: surface reconstruction from large point clouds with a neural kernel field

What NKSR reconstructs, and for whom

NKSR takes a point cloud that has positions and normals and returns an implicit surface you can extract a mesh from. The abstract states the target case plainly: large-scale, sparse, noisy point clouds. That is the shape of data you get from LiDAR sweeps, photogrammetry, or a depth sensor on a moving platform, not the clean synthetic meshes that older reconstruction pipelines assume.

The intended user is someone doing 3D reconstruction work who can read a config file and run a Python script. The repository ships training and inference entry points (train.py, test.py), config trees for ShapeNet, Points2Surf and CARLA, and an examples/ directory with scene-specific scripts such as recons_waymo.py, recons_scannet.py and recons_by_chunk.py. That layout says the authors expect you to reproduce paper results or adapt an existing pipeline, not to drop this into an application as a library with a versioned API.

The abstract claims millions of points in a few seconds and out-of-core handling for very large scenes. Those are the authors' claims about their method. The README separately states the algorithm has been tested on scenes spanning kilometers with millions of points on an RTX 3090 GPU. Both statements are about the authors' setup, not a general guarantee for your hardware.

The kernel field and the sparse solve behind it

NKSR builds on Neural Kernel Fields. The abstract describes the representation as a set of compactly supported kernel functions, which is what makes a memory-efficient sparse linear solver possible. That is the mechanism that separates it from a plain neural field: instead of evaluating a network densely over a volume, the reconstruction is expressed through kernels with finite support, so the linear system stays sparse and scales with the point count rather than the volume.

Noise handling comes from a gradient fitting solve. The abstract lists this as the second change over NKF, alongside scaling to large scenes. The third change is training: the method learns from any dataset of dense oriented points, and the abstract notes you can mix objects and scenes at different scales in one training set.

At inference the Python surface is small. You construct a nksr.Reconstructor on a device, call reconstruct with input positions and normals plus a detail_level, and get back a field. The field exposes extract_dual_mesh, which takes a mise_iter argument. The README's example uses detail_level=1.0 and mise_iter=1. Everything between the point cloud and the mesh (voxel hierarchy, kernel solve, marching cubes style extraction) is inside the compiled extension, which is why the build step matters more than the Python API suggests.

Installing NKSR and running a first reconstruction

The README gives a conda path. Clone the repository, create the environment from environment.yml, activate it, install requirements.txt, then build the package directory. Note the --no-build-isolation flag: without it the build will not see the dependencies already installed in the environment.

git clone git@github.com:nv-tlabs/nksr.git
cd nksr
conda env create -f environment.yml
conda activate nksr
pip install -r requirements.txt
pip install --no-build-isolation package/

The requirements.txt pins torch==2.7.0+cu128 and pulls torch-scatter from the PyG wheel index, plus python-pycg, numpy, matplotlib, ninja, GitPython, pyntcloud and plyfile. The CUDA 12.8 index is not optional here; the news entry from 2025-09-08 states the new wheel is compatible with PyTorch 2.7.0 and CUDA 12.8, and that the full CUDA code was uploaded so users can compile it themselves after the pre-built wheels expired.

Once the build succeeds, the README's own snippet is the shortest path to a mesh. It loads a bunny example, moves positions and normals to the device, and reconstructs.

import torch
import nksr

bunny_geom = load_bunny_example()

input_xyz = torch.from_numpy(np.asarray(bunny_geom.points)).float().to(device)
input_normal = torch.from_numpy(np.asarray(bunny_geom.normals)).float().to(device)

reconstructor = nksr.Reconstructor(device)
field = reconstructor.reconstruct(input_xyz, input_normal, detail_level=1.0)
mesh = field.extract_dual_mesh(mise_iter=1)

What you should see is a field object you can convert to a mesh. This snippet uses the kitchen-sink model, which the README says is released under CC-BY-SA 4.0. For your own data, the README points to NKSR-USAGE.md for data preparation and further examples, and examples/recons_simple.py is the closest thing to a minimal end-to-end script in the repository tree.

Training, checkpoints and the flags that change results

The paper results come from train.py with one config per dataset. The README lists five: configs/shapenet/train_1k_perfect.yaml, train_3k_noise.yaml, train_3k_noiser.yaml, configs/points2surf/train.yaml and configs/carla/train.yaml. Training runs through the Zeus infrastructure, documented in ZEUS_DL.md, which supports tensorboard and wandb. You copy configs/default/zeus.yaml to zeus_config.yaml and edit the wandb account name and the checkpoint and test output directories.

cp configs/default/zeus.yaml zeus_config.yaml
python train.py configs/shapenet/train_3k_noise.yaml

Four flags are worth knowing because they change the model rather than the run. --voxel_size sets the finest voxel level. --feature selects the encoder input: none, normal (the default) or sensor for sensor direction. --tree_depth controls the depth of the sparse feature hierarchy and defaults to 4. --wname only names the wandb run.

Inference works from a checkpoint URL or from your own wandb run. The README shows both forms, including --include to layer a data config on top and --exec udf.enabled=False to override a config value from the command line.

python test.py configs/shapenet/train_3k_noise.yaml --url https://huggingface.co/heiwang1997/nksr-checkpoints/resolve/main/checkpoints/snet-n3k-wnormal.pth --exec udf.enabled=False
python test.py none --ckpt wdb:<WANDB_USER_NAME>/<WANDB_PROJECT>/<WANDB_RUN_ID>

For test.py the README calls out -v or --visualize, --test_print_metrics, and --test_n_upsample with a recommended value of 4. Dataset links are given for ShapeNet (the onet folder goes under ../data/shapenet), Points2Surf, and CARLA. The Points2Surf data was regenerated with blensor to obtain input normals, which is a hint about how much the pipeline depends on normal quality.

Where NKSR is the wrong tool

The build is the first wall. The news entry says the pre-built wheels expired and that the full CUDA code is now shipped so users compile it themselves. That means a working CUDA toolchain, a matching PyTorch, and a build that succeeds against the pinned versions. If your environment is on a different CUDA line, requirements.txt is not going to negotiate. There is no released package on PyPI, so a pip install nksr is not the documented path.

Normals are the second constraint. The default --feature value is normal, and the README's own example passes input_normal. The Points2Surf data had to be regenerated to get input normals. If your point cloud is raw XYZ, you are in the territory the pipeline was not configured for by default, and the README does not describe an unoriented workflow.

Scale cuts both ways. The method is built for large scenes, with out-of-core handling described in the abstract. A single small object with clean geometry is not where this earns its complexity; a classical reconstruction or a simpler fitting approach will be easier to install and debug. And this is research code: the repository has no release tags, the licence is listed as NOASSERTION with the LICENSE.txt being the Nvidia Source Code License, and the README directs business inquiries to an NVIDIA Research licensing form rather than a support channel. If you need a maintained dependency with a support contract, that is a different product category.

How NKSR differs from Poisson and screened Poisson reconstruction

Poisson reconstruction is the obvious comparison because it also takes oriented points and produces a watertight mesh through a global solve. The difference is in what is being solved. Poisson fits an indicator function whose gradient matches the normals, then extracts a level set; the solve is over a fixed octree resolution you choose, and the result is a single global fit. NKSR replaces that with a learned kernel field, so the reconstruction is parameterised by a model trained on dense oriented points rather than by a hand-specified smoothness prior, and the abstract describes the solve as a gradient fitting problem over compactly supported kernels.

In practice that changes three things. Detail level becomes a runtime argument (detail_level, and --voxel_size during training) rather than a fixed octree depth. Noise handling is a trained property, not just a regularisation term. And the method is claimed to handle very large scenes out of core, which a single global Poisson solve on the same data does not do by default.

The trade is dependency weight. Poisson reconstruction is available in libraries you already have. NKSR needs a compiled CUDA extension, a specific PyTorch build, and a checkpoint. If your data is clean and fits in memory, the extra machinery buys you less than it costs.

Maintenance, licence, and what upgrades cost

The repository is not archived, and the last push was on 2026-08-25. The most recent news entry is from 2025-09-08 and concerns the wheel expiry and the CUDA source upload. There are no release tags, so upgrades are tracked by commit rather than by version. That matters for the build: requirements.txt pins torch==2.7.0+cu128 and torch-scatter from a cu128 wheel index, so moving to a newer PyTorch means checking whether the CUDA extension still compiles and whether the pinned wheels still exist.

Licensing is split and worth reading before you ship anything. The repository states it is made available under the Nvidia Source Code License, and the metadata reports the licence as NOASSERTION. The README separately states that the kitchen-sink model is released under CC-BY-SA 4.0. Those are two different terms covering two different things, code and checkpoint, and the README does not reconcile them. It also routes business inquiries through an NVIDIA Research licensing form. This is a description of what the files say, not legal advice; if you plan to use NKSR in a product, read LICENSE.txt and the checkpoint terms yourself.

The practical upgrade cost is a rebuild. Because the extension is compiled from package/ against pinned dependencies, a Python or CUDA bump is a build-and-test cycle, not a dependency resolution. Budget for that whenever you move your environment.

Editorial conclusion

Adopt NKSR if you already have oriented point clouds at scene or object scale and you want a mesh without training a model from scratch; the kitchen-sink model and the Reconstructor API cover that path. Do not adopt it if you need a supported product with a stable wheel, because the README states the pre-built wheels expired and you must compile the CUDA code yourself, and do not adopt it if you cannot supply normals, since the default encoder setting is normal input. Verify first that your CUDA and PyTorch versions match the pinned requirements.txt, that you can build package/ with pip install --no-build-isolation, and which licence applies to the checkpoint you download, since the README says the kitchen-sink model is CC-BY-SA 4.0 while the repository itself carries the Nvidia Source Code License.

Frequently asked questions

Official sources

1. Issues
2. nv-tlabs/NKSR on GitHub
3. Project website
4. README