XCube: Sparse Voxel Hierarchy Diffusion for 1024-cubed 3D Generation

The problem XCube targets: dense voxel grids do not scale to 1024 cubed

A dense voxel grid at 1024 cubed is roughly a billion cells. Storing that as a dense tensor is prohibitive, and generating it in one shot with a diffusion model is worse. XCube's answer, per the abstract, is a hierarchical voxel latent diffusion model that generates progressively higher resolution grids in a coarse-to-fine manner, using a custom framework built on the VDB data structure. The claimed output scale is millions of voxels at a finest effective resolution of up to 1024 cubed, produced feed-forward without test-time optimization. The paper also reports outdoor scenes at 100m x 100m with a voxel size as small as 10cm. The audience is therefore narrow and specific: researchers working on 3D generative modeling who need sparse, high-resolution volumetric output and are willing to build a CUDA stack to get it. This is not a library you add to an application. It is a training and inference codebase tied to a paper.

Coarse-to-fine latent diffusion over VDB, and where mesh extraction sits

The architecture visible in the material is a two-stage pipeline per scale. Stage 1 trains an autoencoder over sparse voxel grids (the configs are named for resolutions such as 16x16x16 and 32x32x32 dense), and Stage 2 trains a latent diffusion model over the autoencoder's latents. Inference then runs diffusion with DDIM sampling and extracts a mesh afterwards. The README is explicit that the released code differs from the paper in two ways: the refinement network is omitted for cleaner code, which the authors say may cause slight variations in results that they describe as not significant, and mesh extraction has been moved out of the VAE into post-processing. That second change matters for anyone reading the paper's pipeline diagram and expecting the VAE to emit geometry. The sparse structure itself comes from fVDB, described in the README as a 3D learning framework, which is installed from a specific OpenVDB pull request branch (pull/1808/head, branch feature/fvdb) rather than from a released package. So the data structure that makes the resolution claim possible is also the least stable part of the install.

Building the environment: conda, a patched OpenVDB, and nksr-cuda

The README states Linux only, and asks for a GPU later than Ampere for fVDB. The setup sequence is: clone the repository, create the conda environment from environment.yml, then build fVDB by cloning AcademySoftwareFoundation/openvdb, fetching pull request 1808 into a local branch, checking it out, replacing fvdb/setup.py with the copy in the repository's assets directory, and running pip install . from inside fvdb. After that, mesh extraction is built separately with python setup.py develop inside ext/nksr-cuda. Docker users are pointed at a base image linked from the fVDB repository and told to apply the same conda setup on top. There is an optional libmamba step for conda solver speed. The presence of a setup.py swap is the detail to notice: the build depends on a file the project ships rather than the one in the upstream branch, which is a common pattern in research code but means the build can break if that branch moves.

Running inference: checkpoints, per-category commands, and visualization scripts

Pretrained checkpoints are hosted on Hugging Face under xrenaa/XCube and are expected in a checkpoints directory. The README says a download script exists at inference/download_pretrain.py but marks it temporarily unavailable, so manual download is the reliable path. ShapeNet inference is per category, for example python inference/sample_shapenet.py none --category chair --total_len 20 --batch_len 4 --ema --use_ddim --ddim_step 100 --extract_mesh, with car and plane variants differing only in the --category value. Waymo unconditional sampling runs through inference/sample_waymo.py with the same sampling flags, and the README marks single-scan conditioning as coming soon, even though the abstract lists scene completion from a single scan among the model's demonstrated tasks. Objaverse text-to-3D runs through inference/sample_objaverse.py. Results are visualized with visualize_object.py or visualize_scene.py, both taking -p for the results path and -i for an id. Training commands in the README uniformly pass --gpus 8, with batch sizes of 32 for autoencoders and 8 with --accumulate_grad_batches 4 for ShapeNet diffusion.

What the release does not give you: data gaps, an omitted network, and a licence that GitHub cannot classify

Three limitations are stated in the material itself. Waymo training data is marked coming soon, so the large outdoor scene capability described in the abstract is not reproducible from the released data links, which cover ShapeNet only. Single-scan conditioning is also coming soon, which removes one of the headline applications from the shipped inference scripts. The refinement network is omitted, so the released pipeline is not the pipeline the paper evaluated, and the authors' own note that differences are not significant is a claim you would have to check on your own data rather than a guarantee. On licensing, GitHub reports NOASSERTION, and the README routes business use to an NVIDIA Research Licensing form. That combination means you should not assume an open-source grant for commercial deployment; read the repository's licence file and, if it matters, talk to the licensing contact rather than treating the GitHub label as a verdict. This is a description of what the material says, not legal advice.

How XCube differs from a mesh-first generator such as GET3D or a NeRF-based pipeline

The closest comparison in the material is SCube, which the news section describes as a NeurIPS 2024 work from the same group that extends XCube on large-scale scene reconstruction. That is a sibling, not a substitute: SCube takes XCube's generative prior toward reconstruction, so if your task is reconstructing a scene from sensor input, SCube is the later line of work and XCube is the foundation it builds on. Against mesh-first generators in the same research space, the difference is representational. A mesh generator must commit to a surface and a topology, which makes thin structures, foliage, and volumetric attributes awkward. XCube's abstract emphasizes sparse voxel grids with arbitrary attributes, and the VDB structure is designed for sparse occupancy rather than a watertight surface. The cost of that choice is that your output is a voxel grid, and getting a mesh out requires the separate nksr-cuda extraction step. If your downstream consumer wants a clean triangle mesh, the voxel representation is an intermediate you have to pay to convert.

Maintenance and upgrade cost of a pinned research stack

The install depends on a pull request branch of OpenVDB, a repository-local replacement for fvdb/setup.py, a separate CUDA extension in ext/nksr-cuda, and a conda environment file. Each of those can drift independently of XCube. There are no releases retrieved for this repository, so there is no tagged version to pin against; you pin against a commit on main. The README's own note that the released code differs from the paper is a reminder that the artifact is not versioned against a publication. The practical implication is that reproducing a run months later means archiving the exact OpenVDB commit you fetched, the assets/setup.py you copied, and your conda environment, because the upstream branch feature/fvdb is not under this project's control. The repository is not archived and was pushed recently, but nothing in the material describes a support commitment, a deprecation policy, or a changelog.

Editorial conclusion

Adopt XCube if you have Ampere-or-newer GPUs, a Linux box, and a reason to generate or complete large sparse voxel scenes rather than meshes. Do not adopt it if you need a supported package with a clean licence statement, or if you only have a single consumer GPU: the README's training commands assume 8 GPUs, and the fVDB build requires a GPU later than Ampere. Before committing, verify three things against the repository itself: the actual licence terms, since GitHub reports NOASSERTION; whether the Waymo data and single-scan conditioning are still marked coming soon; and whether the refinement network omission changes results on your own category.

Official sources

1. Issues
2. nv-tlabs/XCube on GitHub
3. Project website
4. README