SAGA (Segment Any 3D GAussians): 3D segmentation on a pre-trained Gaussian Splatting scene
The official implementation of Segment Any 3D GAussians (AAAI-25)
At a glance
- What is it?
- SAGA is the AAAI-25 reference implementation that attaches SAM masks and contrastive 3D features to an existing 3D Gaussian Splatting model, then segments objects interactively in a Dear PyGui window. It assumes you already trained the Gaussians and have a CUDA-capable GPU.
- Who is it for?
- Adopt SAGA if you already have a trained 3DGS model on a 360_v2, nerf_llff_data or LERF scene and need to pull a single object out of it, either through saga_gui.py or prompt_segmenting.ipynb, and you accept that the pipeline starts with extract_segment_everything_masks.py plus get_scale.py before any segmentation is possible.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 48 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What SAGA adds to a Gaussian Splatting scene
A trained 3D Gaussian Splatting model is a cloud of anisotropic Gaussians with colors and opacities. It renders well. It has no notion of objects, so there is no way to ask for the fern and not the pot it sits in. SAGA is the official implementation of the paper Segment Any 3D GAussians, and it answers that question by giving every Gaussian an affinity feature that can be compared against point prompts.
The audience is narrow and the README says so implicitly: the installation is described as similar to 3D Gaussian Splatting, the data section names 360_v2, nerf_llff_data and LERF, and the first training step is the 3DGS training script. This is a research tool for people who already work with Gaussian Splatting reconstructions. It is not a viewer you point at a video and get masks out of.
How the affinity features and mask scales are built
The pipeline has three stages, and the ordering is not optional. First, the Gaussians are trained with train_scene.py, which the README describes as inheriting all attributes from 3DGS. Second, extract_segment_everything_masks.py runs SAM over the scene images and writes sam_masks; get_scale.py then reads the pre-trained 3DGS model and writes mask_scales. The README explains why the model has to exist before the scales: the pre-trained 3D-GS model is needed for mask scales extraction.
Third, train_contrastive_feature.py trains the 3D affinity features that the segmentation step compares against. The README's example passes --iterations 10000 and --num_sampled_rays 1000, so the feature training is a separate optimization over sampled rays rather than a post-hoc projection of 2D features.
At inference the GUI exposes two knobs that matter more than the rest. The scale parameter is used for both segmentation and clustering, and score thresh controls the similarity threshold used for segmentation. The README warns that the 2D preview and the 3D result can disagree, which is a fair description of any method that lifts 2D masks into a 3D representation.
Installing SAGA and running a first segmentation
The README gives the clone commands and then a conda environment file. The environment is named gaussian_splatting, which is a leftover from the upstream project rather than a SAGA-specific name.
git clone https://github.com/Jumpat/SegAnyGAussians.git
conda env create --file environment.yml
conda activate gaussian_splattingThe default segmentation model is the public ViT-H SAM checkpoint. The README says to download it and place it under ./third_party/segment-anything/sam_ckpt. After that, the scene data has to match the layout the README prints, with images, sparse, features, sam_masks and mask_scales directories under each scene.
Before any segmentation you need Gaussians and the derived masks. These three commands are the minimum sequence, and the downsample flag is the one to tune when memory runs out.
python train_scene.py -s <path to COLMAP or NeRF Synthetic dataset>
python extract_segment_everything_masks.py --image_root <path to the scene data> --sam_checkpoint_path <path to the pre-trained SAM model> --downsample <1/2/4/8>
python get_scale.py --image_root <path to the scene data> --model_path <path to the pre-trained 3DGS model>Feature training comes next, then the GUI. The README's own example uses 10000 iterations.
python train_contrastive_feature.py -m <path to the pre-trained 3DGS model> --iterations 10000 --num_sampled_rays 1000
python saga_gui.py --model_path <path to the pre-trained 3DGS model>In the window, right click places point prompts once segmentation mode is checked, left drag rotates and mid drag pans. Clicking segment3D produces the 3D result, roll back undoes it, and save as writes ./segmentation_res/your_name.pt, described in the README as a binary mask over all 3D Gaussians. Rendering that mask back onto training views uses render.py with --precomputed_mask and --target scene --segment.
Where SAGA gets expensive or simply refuses to fit
The README is direct about one constraint: sometimes the downsample is essential due to the limited GPU memory. That is a real ceiling. SAM ViT-H over a full image set is the memory-heavy step, and reducing the downsample factor trades mask resolution for the ability to finish at all.
The clustering path has a second, different cost. The README notes that clustering the whole scene may take a while because HDBSCAN is used without GPU support. That is a deliberate convenience choice, and it means cluster3d on a large scene is a CPU-bound wait rather than an interactive operation.
Open-vocabulary segmentation is the sharpest limitation. The README states that it is only implemented in the jupyter notebook, so a user who wants text-driven queries cannot stay in the GUI. There is also a documented inconsistency between the 2D preview and the 3D result, which matters if your workflow depends on trusting the preview before committing to a segmentation.
Finally, SAGA is the wrong tool if you have no Gaussians. Everything downstream depends on a pre-trained 3DGS model, including the mask scales. If you only have images, this is not the entry point.
SAGA against 2D-mask lifting and language-feature methods
The related searches around this project cluster into two families, and the difference between them is architectural rather than cosmetic.
SA3D and Gaussian Grouping are the closest neighbours in the lifting family: they also start from SAM masks and propagate them into a 3D representation. SAGA's distinguishing choice is that it does not stop at propagating masks. It trains a separate contrastive feature field with train_contrastive_feature.py and segments by comparing features against point prompts, with the scale and score threshold exposed in the GUI. That gives an interactive loop, at the cost of an extra training run per scene.
LangSplat and Feature 3DGS sit in the language-feature family. They aim at text queries over the scene. SAGA does support CLIP features through get_clip_features.py, but the README confines open-vocabulary segmentation to the notebook. So if text-driven queries are the primary requirement, a method built around language features from the start is a better fit, and SAGA is the better fit when the requirement is precise interactive object selection with a rollback button.
Maintenance, upgrades and the Apache-2.0 licence
The repository is not archived, and the last push to the v2 branch was on 2026-08-02. No releases were retrieved, so there is no tagged version to pin against; you are tracking a branch. That has a practical consequence for upgrades: there is no changelog to read before pulling, and the data layout the README prints is the contract you have to keep stable in your own scripts.
The default branch is v2, not main, so a clone that does not specify a branch may land somewhere other than the code the README describes. The conda environment file is the other upgrade surface. Because the project inherits from 3D Gaussian Splatting and uses submodules and a third_party directory, a change in the upstream 3DGS code or in the SAM checkpoint path can break the extract and scale steps before you reach segmentation.
The repository carries an Apache-2.0 LICENSE at its root. That is the licence text you need to read yourself before redistributing anything, since the project also depends on components with their own terms, including the SAM checkpoint and the datasets it names. Nothing here is legal advice; the point is that the Apache-2.0 file covers this repository, not necessarily everything it downloads.
Editorial conclusion
Adopt SAGA if you already have a trained 3DGS model on a 360_v2, nerf_llff_data or LERF scene and need to pull a single object out of it, either through saga_gui.py or prompt_segmenting.ipynb, and you accept that the pipeline starts with extract_segment_everything_masks.py plus get_scale.py before any segmentation is possible. Skip it if you want a standalone segmentation tool that runs on a laptop without CUDA, or if you cannot retrain the Gaussians, because the mask scales are derived from a pre-trained 3DGS model. Before committing, verify three things: that your GPU memory survives the SAM mask extraction at your chosen downsample, that the open-vocabulary path through get_clip_features.py is acceptable to you even though the README says it lives only in the notebook, and that the Apache-2.0 LICENSE in the repository root is compatible with how you plan to ship the result.
Frequently asked questions
What is SAGA (Segment Any 3D GAussians) and what does it do?
It is the official implementation of the AAAI-25 paper Segment Any 3D GAussians. It trains 3D Gaussian affinity features on top of a pre-trained 3D Gaussian Splatting model so that point prompts can select objects in 3D.
How do I install SAGA and run it for the first time?
Clone the repository, create the conda environment from environment.yml and activate gaussian_splatting, then download the ViT-H SAM checkpoint into ./third_party/segment-anything/sam_ckpt. After training the Gaussians, run extract_segment_everything_masks.py and get_scale.py, then start saga_gui.py with --model_path.
Does SAGA support open-vocabulary segmentation in the GUI?
No. The README states that open-vocabulary segmentation is temporarily implemented only in the jupyter notebook, and points to prompt_segmenting.ipynb for instructions. The GUI covers point-prompt segmentation and clustering.
Why does SAGA need a pre-trained 3DGS model before extracting masks?
Because the mask scales are derived from that model. The README says the pre-trained 3D-GS model is needed for mask scales extraction, which is why get_scale.py takes --model_path pointing at the trained Gaussians.
What can I do if SAGA runs out of GPU memory?
The README notes that sometimes the downsample is essential due to the limited GPU memory, and the extract_segment_everything_masks.py example accepts --downsample with 1, 2, 4 or 8. Lower mask resolution is the trade for fitting in memory.
Community notes