ReconViaGen: reconstructing 3D objects from multiple views by generating instead of fusing

The method reframes reconstruction as a generation problem

ReconViaGen is described as a project for accurate multi-view 3D object reconstruction via generation, and the repository is the code release for a paper accepted at ICLR 2026, with the arXiv preprint linked as 2510.23306.

The framing is the interesting part. Rather than fusing several views of an object into a single reconstruction by combining their evidence, the method treats the problem as generation, and the title says toward accurate multi-view 3D object reconstruction via generation. The topics confirm the shape: 3d, 3d-generation, 3d-reconstruction, image-to-3d and pose-estimation.

The pipeline has two stages that correspond to two data representations, and the training scripts are named after them. There is a train_ss.sh for the SS Stage flow model and a train_slat.sh for the SLat Stage flow model, and the dataset has a matching split with a renders_random_env/ directory of multi-view images and an lh-slats/ directory of structured latents.

The inference side has the same two-level structure. app.py reconstructs an object without refinement, and app_fine.py reconstructs it with refinement, with the README giving VRAM figures for each: reconstructing 16 images without refinement consumes less than 18GB of VRAM, and with refinement less than 24GB.

The primary language is Python, the licence is MIT, and the project is from GAP-LAB-CUHK-SZ. The homepage is a project page, and the last push to the repository was on 2026-04-04 with no GitHub releases, which is consistent with a research repository that publishes code on branches rather than cutting tagged versions.

Main holds v0.2, and the v0.5 branch is where the newer work lives

The release structure is the first thing to understand, and the README's news section spells it out.

The v0.2 release put the training and inference code in the main branch, along with an optimisation of the inference process, and that is what the VRAM figures above refer to. There is also a community note recording that an unofficial implementation of ReconViaGen has been released by another developer, which is a useful data point about how much of this area people are reimplementing.

The v0.5 release is on the v0.5 branch, and the README is explicit that it contains the inference code for ReconViaGen-v0.5. The stated contribution is a multi-view fusion strategy proposed for TRELLIS.2, with ReconViaGen then combined with TRELLIS.2 to enable generation of high-resolution meshes with physically based rendering.

The dependency on TRELLIS.2 is credited to that project explicitly, with a thank you for the work. This is a composition, not a replacement: the v0.5 path borrows TRELLIS.2 and adds the multi-view conditioning.

The bug fix note is also worth reading. Some bugs have been fixed in the v0.5 branch and the README asks that people resynchronise with the remote repository, and a demo has been deployed as a Hugging Face Space for that version.

So the repository has three distinct states rather than one: the v0.2 code on main, the v0.5 inference code on the v0.5 branch, and an alpha demo running on Hugging Face Spaces. The README calls the demo an alpha demo and asks for feedback on failure cases to help improve the model, which is a reasonable thing to ask of a research release.

Installing means a recursive clone and a scripted conda environment

The installation is two steps, and both are commands you can copy directly.

First the clone, and the --recursive flag matters because the repository carries submodules.

git clone --recursive https://github.com/GAP-LAB-CUHK-SZ/ReconViaGen.git
cd ReconViaGen

Then the environment, created by a setup script with a set of feature flags. The command creates a new conda environment named reconviagen and installs the dependencies, with pytorch 2.4.0 and CUDA 12.1 as the stated base.

. ./setup.sh --new-env --basic --xformers --flash-attn --spconv --mipgaussian --kaolin --nvdiffrast --demo

The flags are the readable summary of what the project needs. --basic is the base install, xformers and flash-attn are the attention implementations, spconv is the sparse convolution library, mipgaussian and kaolin and nvdiffrast are the graphics dependencies, and --demo pulls the extra pieces the demo needs.

The local demo is then two scripts, one for each inference path.

python app.py

python app_fine.py

The requirements.txt in the repository gives the pinned versions, and it is worth reading because several of the entries are not ordinary PyPI packages. There is an extra index URL for the CUDA 12.1 build of PyTorch, and three entries are direct wheel URLs rather than version specifiers: a flash-attention wheel pinned to torch 2.4 with a specific ABI, and two wheels pulled from a Hugging Face Space for TRELLIS, one for the diff_gaussian_rasterization package and one for nvdiffrast. Those are the compiled graphics pieces, and the fact that they are fetched from a Space rather than a package index tells you the dependency graph is pinned tightly to one CUDA and Python combination.

The dataset pairs camera metadata, RGBA renders and structured latents

The dataset documentation is the clearest description of what the model actually consumes, and the layout is worth reading carefully because the training data format is the interface between the cameras and the geometry.

The processed dataset is downloadable from Hugging Face as ProObjaverse-300K, and it is organised in two halves. The renders_random_env/ directory holds shards, and each shard holds one.tar per object. Inside a tar, a directory per view holds a numbered.json with the camera metadata and a numbered.rgba.webp with the RGBA render at 1024 by 1024, up to around 80 views per object.

The lh-slats/ directory holds shards of.npz files, one per object, and each npz has two arrays. feats is float32 with shape N by 8, described as the latent features per voxel, and coords is uint8 with shape N by 3, described as the voxel coordinates in the range 0 to 63.

The pairing mechanism is the uid, which is shared between the render tar and the slat npz and is used to pair them at training time. The camera file contains the pose of every rendered view, with the documentation noting the extrinsic is 4x4 and the intrinsic 3x3.

So the input to the model is a set of posed images and the target is a voxelised latent field, and the camera metadata is what makes the multi-view part work. A reconstruction method that ignored camera extrinsics would be guessing at geometry, and this dataset structure makes the poses a first-class part of the data rather than an afterthought.

The renders are random-environment renders, which suggests a deliberate choice to widen the appearance distribution the model sees.

Two training scripts, eight A100s, and a Gradio checkpoint tester

Training is described in three numbered steps, and the hardware note is the same for both stages: the model was trained with 8 A100 GPUs of 80GB each.

The first step trains the flow model of the SS Stage on ProObjaverse-300K.

. ./train_ss.sh

The second trains the flow model of the SLat Stage, also on the same dataset.

. ./train_slat.sh

The third is the one you will actually want, which is trying the trained checkpoints in an interactive interface rather than only through the batch demo scripts.

python app_try.py --ss_ckpt /path_to_your_trained_ss_ckpt --slat_ckpt /path_to_your_trained_slat_ckpt

So the checkpoint interface is two paths, one per stage, which is the same two-stage structure the dataset and the inference scripts follow. The scripts are thin wrappers, and the repository also holds train_slat.py and train_ss.py directly, so you can read the Python to see what the shell is doing.

The root listing shows the rest of the code. app.py, app_fine.py and app_try.py are the three entry points, dataset.py handles the data, and the trellis/ directory holds the TRELLIS integration that the v0.5 work builds on. The extensions/ and wheels/ directories are there for the compiled dependencies, and requirements.txt pins the Python side.

One detail worth flagging for anyone planning to reproduce this: the paper's own citation is included as a BibTeX block in the README, and it lists the arXiv preprint number with a 2025 year, while the repository description says ICLR 2026. The preprint and the conference version are therefore the same work, and the arXiv entry is the citable one until the conference proceedings are indexed.

Where ReconViaGen is the wrong tool

The first wrong-tool case is a production service that needs a stable, versioned model endpoint. There are no GitHub releases, the newest work lives on a branch, the demo is labelled an alpha demo, and the README asks for feedback on failure cases. That is a research release, and the branch structure means the code you clone depends on which branch you pick.

The second case is a machine that does not fit the hardware profile. The README states 8 A100 GPUs of 80GB for training, and 18GB and 24GB of VRAM for the two inference paths with 16 images. A single consumer GPU will not run the multi-view path as documented, and the setup script's flags for xformers, flash attention, spconv and the rasterisation wheels are pinned to one CUDA and Python combination, so a mismatched environment is a real possibility rather than a hypothetical.

The third case is a task that is not multi-view reconstruction. Despite the name, this is about reconstructing an object from several posed views of it, with the camera metadata as a first-class input. A single image to 3D task is a different method, and the model here is not the tool for it.

The fourth case is a user who needs the newest result without assembling a research environment. The Hugging Face Space demo is the way in, and for evaluating the method it is likely enough.

One more practical point: the dependency on TRELLIS.2 in the v0.5 path is a dependency on another research codebase, and the compiled wheels for it are fetched from a Hugging Face Space rather than from a package index. That is a supply chain detail rather than a criticism, and it is exactly the kind of thing to check before running untrusted code on a machine, since the install step fetches binaries from a space URL rather than resolving a version from an index.

Editorial conclusion

ReconViaGen suits a researcher who wants to reproduce or extend a multi-view reconstruction method and is comfortable with a two stage flow model setup on multiple high memory GPUs. It does not suit a production service, because the model is a research release distributed by branch, with the newer capability living on a v0.5 branch rather than on main. Before running it, read the v0.5 branch notes, since the README says the main branch holds v0.2 and the newer inference code is a separate branch, and check the VRAM figures, which the README states as under 18GB for the plain path and under 24GB with refinement, against the hardware you have.

Frequently asked questions

Official sources

1. GAP-LAB-CUHK-SZ/ReconViaGen on GitHub
2. Issues
3. License: MIT
4. Project website
5. README