S2AM3D segments 3D point clouds at a granularity you choose

The problem the paper names: parts are not objects

The abstract states the two failures the authors set out to fix. Native 3D models lack fine-grained part understanding, while approaches that lean on 2D priors produce inconsistent predictions across views. S2AM3D is the answer they propose: 2D segmentation priors combined with 3D consistent supervision. The two named components are where the design lives. A point-consistent part encoder aggregates multi-view information while respecting the 3D consistency of individual points, so a point that is the same point in every view stays the same prediction. The decoder adds a controllable part decoder with a learnable weight generation module, which is what allows segmentation granularity to be scaled at runtime rather than fixed when the model was trained.

Granularity as a runtime knob instead of a training-time choice

Most part segmentation models answer one question at one resolution: is this region of the mesh a leg, a handle, or a shell. The learnable weight generation module in the controllable part decoder changes that, letting the same model operate at a coarser or finer granularity depending on what the user asks for. That matters for a practical reason. Annotating a dataset at three levels means training and maintaining models that agree with each other, and disagreement between them at inference is a real source of noise. A single model with a scale input avoids that by construction. The abstract claims leading performance across multiple evaluation settings and particular stability on complex structures, but those are the authors' claims about their own experiments, and the repository does not publish a benchmark table you can check against a competing method.

The dataset is the larger contribution

The dataset is described as built by an automated data processing pipeline, collecting over 100,000 point cloud instances across 400 categories, annotated with approximately 1.2 million fine-grained part labels at three granularity levels. The README calls it the most comprehensive dataset for part-level point cloud segmentation to date, which is a claim about coverage rather than about a measured result, so treat it as the authors' framing. It is hosted on Hugging Face Datasets under the HITshan organisation, and the pretrained models sit alongside it on the Hugging Face Hub. The three granularity levels match the paper's premise, since a model that scales its granularity is only as useful as the labels it was trained against.

Installing the environment, which is the real work

The recommended path is the conda environment file:

conda env create -f environment.yml
conda activate s2am3d

The manual path spells out every step, and the pinned versions are the point. The environment is created with Python 3.10, CUDA 12.4 is installed from the NVIDIA channel, and PyTorch is pinned to an exact release against a specific wheel index:

pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124

torch-scatter then has to match that PyTorch build exactly:

pip install torch-scatter -f https://data.pyg.org/whl/torch-2.4.0+cu124.html

Getting that pairing wrong is the most common way this install fails, since the wheel index is indexed by torch version and CUDA version together. The rest of the dependency list is long, and each entry is a real dependency rather than filler: psutil, lightning and h5py for training infrastructure, yacs and omegaconf for configuration, trimesh, scikit-image, mesh2sdf, tetgen, pymeshlab, plyfile and open3d for geometry, viser for visualisation, and polyscope, potpourri3d, einops, libigl, simple_parsing and arrgh for rendering and configuration. A Linux user also needs libx11-6, libgl1 and libxrender1, and vtk is installed last.

Placing the checkpoints and the expected layout

Pretrained models come from the Hugging Face Hub and go into a ckpt/ folder that you create yourself:

mkdir -p ckpt
# Download the following files and place them in ckpt/
# - Encoder.ckpt
# - S2AM3D_decoder.pt

The file names matter, because the encoder and the decoder are separate checkpoints and the demo will look for both. The repository is laid out with a ckpt/ directory at the top, then decoder/, encoder/, demo/ and further entries, and the training/ and data_preprocessing/ directories sit alongside those. The TODO list in the README is fully checked: training code is released, more input formats are supported, and the data preprocessing scripts are published. That matters for a research user, because preprocessing scripts are normally the missing piece that stops a dataset from being usable.

The interactive demo is the fastest way to see it work

The demo runs a server rather than a script that prints to the terminal:

cd decoder
bash run_interactive_demo.sh

Once it starts, the README says to open a browser and go to:

http://localhost:8080

The demo directory ships two npy sample files, which are the only inputs the README names. The choice of a browser-based viewer is sensible for this kind of model, since the point of the work is to look at segmentation output on a shape, and a viewer that can show a point cloud and its per-part colouring is more useful for judging a result than a saved mesh. The tradeoff is that you now have a local service running on a fixed port, and the README does not describe how to change that port.

Where this repository falls short for production use

Three limits, all of them typical of a paper implementation. The first is release discipline: the repository publishes no GitHub releases, the last push was on 2026-07-21, and the version is only the arXiv identifier 2512.00995, so there is no upstream version to pin against. The second is the environment, which is pinned to Python 3.10, CUDA 12.4 and PyTorch 2.4.0, and PyTorch 2.4.0 is far enough back that a production image will want something newer. The third is documentation of the interface itself. The README covers installation, the dataset, the demo and the citation, and it does not document the input format expected by the encoder, the shape of the granularity parameter, or the inference API beyond the shell script. On the positive side, the licence is MIT and the pretrained weights and dataset are published, and the work is acknowledged as building on PartField, OpenShape and Segment Anything.

Editorial conclusion

S2AM3D is a good fit for a research group that needs part-level labels on 3D shapes with a granularity it chooses at inference time, and that has a GPU environment and a tolerance for a research codebase. It is a poor fit for a production pipeline that needs a stable, supported library, because the repository is a paper implementation with no releases, and it is a poor fit for a machine without the CUDA 12.4 and PyTorch 2.4 stack the manual install assumes. Check three things before you invest an afternoon: that the checkpoint files land in ckpt/ with the exact names Encoder.ckpt and S2AM3D_decoder.pt, since the demo expects that layout, that your environment can satisfy the pinned torch 2.4.0 install and the torch-scatter wheel for that exact version, and that the point cloud you feed is in the format the demo's npy files use, because the repository documents the dataset and the pretrained models but not the input format in the README.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. sumuru789/S2AM3D on GitHub