Pix2Vox: reconstructing a 3D shape from one or several images

A paper implementation, not a productised library

Pix2Vox is the official implementation of the paper Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images, published at ICCV in 2019. The authors are Xie Haozhe, Yao Hongxun, Sun Xiaoshuai, Zhou Shangchen and Zhang Shengping, and the paper is available on arXiv.

The framing matters for what you get. This is a research code release, and the repository reads like one: a runner.py entry point, a config.py you edit by hand, a models/ directory, a datasets/ directory and a core/ directory with the network definitions. There is no package on PyPI, no typed API, and no GitHub releases.

The problem it addresses is 3D reconstruction from images. Given one picture of an object, or several pictures of it, the model predicts a 3D shape. The output is a voxel grid rather than a mesh, and the README makes the data assumption explicit by pointing at ShapeNet voxelized models in .binvox format, so the representation is a coarse volumetric grid rather than a surface.

The word context-aware in the title refers to the part of the architecture that is the actual contribution. A single-view reconstruction has to guess the unseen sides of the object, and the method's premise is that prior knowledge about what kinds of objects exist helps it. The follow-up work, Pix2Vox++, is described as multi-scale context-aware reconstruction from single and multiple images and was published in the International Journal of Computer Vision.

The audience is a researcher or student who wants this specific method's code, or wants to extend it. The homepage of the project points at the author's project page.

One honest note: the README lists the follow-up paper but the repository is the implementation of the original work, so anyone expecting the multi-scale variant should read the repository's branches and the follow-up's own release before assuming it is here.

The repository layout and the two variants

The top-level listing is compact and tells you the shape of the code: .gitignore, .style.yapf and .yapfignore for formatting, LICENSE, README.md, config.py, then core/, datasets/, models/, requirements.txt, runner.py and utils/.

That division is the standard one for this kind of release. config.py holds the global configuration object, the __C convention visible in the dataset paths. models/ holds the network definitions for the different variants, datasets/ holds the loaders, core/ holds the shared pieces such as losses and network blocks, and utils/ holds helpers. runner.py is the single entry point that does everything, training and testing alike, selected by a flag.

There are two model variants and they are not in the same place. The pretrained models on ShapeNet are published as Pix2Vox-A at 457.0 MB and Pix2Vox-F at 29.8 MB, which is a large difference in size and signals a very different computational cost between them. The README states that to train or test Pix2Vox-F you need to check out the Pix2Vox-F branch first, which means the variants are separate lines of development rather than a runtime option:

git checkout -b Pix2Vox-F origin/Pix2Vox-F

The command creates a local branch tracking the remote Pix2Vox-F branch. So the choice of variant is a choice of branch, decided before anything else.

The formatting configuration is worth noting for anyone planning to contribute. .style.yapf and .yapfignore indicate the project uses yapf, which is the formatter referenced by the style file naming, and the CI signals in the README are a SonarCloud quality gate badge and a CodeFactor badge, so style compliance is checked automatically.

Cloning, dependencies and the config.py paths you must edit

Setup is three steps, and the third is the one that catches people.

First clone the repository:

git clone https://github.com/hzxie/Pix2Vox.git

Then install the Python dependencies from the requirements file:

cd Pix2Vox
pip install -r requirements.txt

The requirements list is short and gives a good picture of the stack: argparse, easydict, matplotlib, numpy, opencv-python, scipy, torchvision and tensorboardX. torchvision is doing the heavy lifting for the image encoder, easydict is what makes the dotted config attribute access work, and tensorboardX is the training logger.

Then you have to edit config.py, because the dataset paths are placeholders. The README is explicit that you need to update the file path of the datasets, and the lines look like this:

__C.DATASETS.SHAPENET.RENDERING_PATH        = '/path/to/Datasets/ShapeNet/ShapeNetRendering/%s/%s/rendering/%02d.png'
__C.DATASETS.SHAPENET.VOXEL_PATH            = '/path/to/Datasets/ShapeNet/ShapeNetVox32/%s/%s/model.binvox'
__C.DATASETS.PASCAL3D.ANNOTATION_PATH       = '/path/to/Datasets/PASCAL3D/Annotations/%s_imagenet/%s.mat'
__C.DATASETS.PASCAL3D.RENDERING_PATH        = '/path/to/Datasets/PASCAL3D/Images/%s_imagenet/%s.JPEG'
__C.DATASETS.PASCAL3D.VOXEL_PATH            = '/path/to/Datasets/PASCAL3D/CAD/%s/%02d.binvox'
__C.DATASETS.PIX3D.ANNOTATION_PATH          = '/path/to/Datasets/Pix3D/pix3d.json'
__C.DATASETS.PIX3D.RENDERING_PATH           = '/path/to/Datasets/Pix3D/img/%s/%s.%s'
__C.DATASETS.PIX3D.VOXEL_PATH               = '/path/to/Datasets/Pix3D/model/%s/%s/%s.binvox'

The templates are worth reading closely, because they tell you the directory layout the loaders assume. The ShapeNet rendering path is a three-level template ending in a numbered PNG, the ShapeNet voxel path ends in model.binvox, the PASCAL3D paths use an imagenet subfolder and .mat annotations, and the Pix3D paths use a pix3d.json annotation file with images in an img directory and models under model. Notice that PASCAL3D is configured here even though the README's dataset section only links ShapeNet and Pix3D downloads, so the code carries support for a third dataset that the download instructions do not cover.

The .binvox extension is the giveaway that the ground truth is a voxel grid, and every dataset path encodes the same expectation about how the ground truth is stored.

Training, testing and the pretrained checkpoints

Once the paths are right, both operations are a single command through runner.py.

Training is:

python3 runner.py

Testing is the same entry point with a test flag and a weights path:

python3 runner.py --test --weights=/path/to/pretrained/model.pth

The two commands share the runner, which is typical of this style of research release. The same config drives the data loaders, the model construction and the optimisation, and the flag only decides which loop runs.

The pretrained ShapeNet models are the shortcut for anyone who does not want to train. The README links Pix2Vox-A on ShapeNet at 457.0 MB and Pix2Vox-F on ShapeNet at 29.8 MB, both served from a gateway URL.

That size difference is the first practical signal about the two variants. The A variant is roughly fifteen times the size of F, which usually reflects a much larger network and a correspondingly heavier training and inference cost. The README does not describe the internal difference between the variants, so the details live in the models/ directory of the branch you check out.

The datasets themselves are linked from the README. ShapeNet rendering images are a .tgz archive, ShapeNet voxelized models are a separate .tgz, and Pix3D images and voxelized models come as a .zip. The ground truth for both is voxelised, matching the .binvox paths in config.py.

Why the voxel representation is the main limitation

The design decision that shapes everything downstream is the output representation, and it is worth being blunt about the cost.

A voxel grid spends memory on empty space. A 32 cubed grid, which is what the ShapeNetVox32 dataset name implies, is a coarse shape. The visual result is a blocky approximation of an object rather than a smooth surface, and the quantity of voxels needed to improve the resolution grows with the cube of the resolution. A model that produces finer voxels needs more memory, more compute, and a bigger training set.

That is the trade-off the paper made, and it is a sensible research trade-off, since voxel predictions lend themselves to straightforward end-to-end training, whereas mesh supervision needs extra machinery. But it means this repository cannot give you a production asset. A game studio or a 3D printing workflow needs a watertight mesh with a texture, and the pipeline from a voxel grid to a usable mesh is a separate piece of work that this repository does not contain.

There is a second limitation that follows from the same place. The method predicts from images of objects that are roughly centred and photographed, using datasets of rendered and real product images. Generalisation to a scene, to a person, to an occluded object in a cluttered room, or to an image from an arbitrary viewpoint is not something the setup addresses, and the paper's own framing, context-aware 3D object reconstruction, is about objects rather than scenes.

A third practical limit is the dependency set. The requirements list pins nothing to a version, so a fresh install in a few years will pull whatever the current releases of numpy, scipy, opencv-python and torchvision are, and older research code tends to break on numpy or torchvision API changes. If you hit an import error, that is the likely cause.

The MIT licence and a repository with no releases

The licensing is the easy part. The project is open sourced under the MIT licence, stated in the README and with a LICENSE file at the root. MIT is permissive, with no copyleft obligation, so commercial use of the code and the weights is not restricted in the way a copyleft licence on the model would be.

That said, the licence covers the code, and the trained weights on a research dataset are a separate question that the README does not address. ShapeNet and Pix3D have their own terms of use, and the images and voxel models it points at are not yours to redistribute just because the code that reads them is MIT. If you plan to redistribute data or train on a dataset you do not have a licence for, that is a separate conversation from the code licence.

The maintenance picture is thinner. There are no GitHub releases, so there is no versioned artifact to pin and no changelog of published versions to check. The last push was on 2026-06-15.

What that means in practice is that you should treat the repository as a fixed snapshot. If you need the follow-up method, look for Pix2Vox++ separately, since the README only points at the paper for that work and not at an implementation in this tree. If you need to cite the method in your own work, the README gives a BibTeX entry for the ICCV 2019 paper, with the five authors and the booktitle ICCV, and that is the citation to use.

Editorial conclusion

Adopt Pix2Vox if you want to reproduce or build on the specific 2019 ICCV work on context-aware 3D reconstruction from images, and you have a GPU, the ShapeNet or Pix3D data, and the patience to edit config.py by hand. Do not adopt it for a production 3D pipeline, since it outputs coarse voxels and has no published release, and do not expect the follow-up method to be in this repository. Verify first that your dataset paths in config.py match the layout the loaders expect, that you pick the checkpoint matching your variant, and that the memory footprint of the 457.0 MB Pix2Vox-A weights fits the machine you are training on.

Frequently asked questions

Official sources

1. hzxie/Pix2Vox on GitHub
2. Issues
3. License: MIT
4. Project website
5. README