Inpaint-Anything: click an object, remove or refill it with SAM and LaMa

The gap Inpaint-Anything fills between segmentation and inpainting

Segmentation models tell you where an object is. Inpainting models fill a masked region. Neither one, on its own, gives you a usable object-removal tool, because the mask has to come from somewhere and has to be accurate at the boundary. Inpaint-Anything is the glue: the user clicks a point on an object, SAM turns that click into a mask, and an inpainting model fills the masked hole. The README describes the same three steps for Remove Anything: click on an object, SAM segments the object out, inpainting models such as LaMa fill the hole.

The audience is narrower than the demo GIFs suggest. This is a repository of Python entry points (remove_anything.py, fill_anything.py, replace_anything.py, remove_anything_video.py, remove_anything_3d.py) plus a local web UI under app/. It suits researchers and engineers who want the pipeline as code they can modify, or who want to batch-process a folder of images. It is a poor fit for someone who wants a polished interactive editor with layers, undo and brush refinement; the README's own framing is a click-to-remove demo, not an editing suite.

The feature matrix in the README is explicit about what is finished. Remove Anything, Fill Anything, Replace Anything, Remove Anything 3D and Remove Anything Video are checked. Fill Anything 3D, Replace Anything 3D, Fill Anything Video and Replace Anything Video are unchecked, meaning the video and 3D paths are removal-only. If your task is replacing a background in a video, the repository does not claim to do it.

How the click-to-mask-to-inpaint pipeline is wired

The architecture is a sequence of separate models rather than one end-to-end network, and the repository layout reflects that. segment_anything/ holds a vendored copy of SAM, lama/ holds the LaMa inpainting code and its requirements file, sttn/ holds the video inpainting model, pytracking/ and ostrack.py cover tracking, nerf/ covers the 3D path, and stable_diffusion_inpaint.py wraps the text-guided inpainting model.

Data flows in one direction. A click or a set of points goes into SAM, which produces a binary mask. That mask, plus the original image, goes into the inpainting backend, which returns the filled image. For text-guided operations the mask and a text prompt go into Stable Diffusion instead of LaMa, which is why Fill Anything and Replace Anything depend on a different backend than Remove Anything. For video, the README lists OSTrack for tracking and STTN for video inpainting, so the mask is propagated across frames by the tracker before the inpainting stage runs. The 3D path uses NeRF-based code in nerf/ and is exposed through remove_anything_3d.py.

Because the stages are separate, each one can be swapped or skipped. That is the practical advantage of this design and also its main operational cost: every stage brings its own checkpoint, its own dependency set and its own failure modes. A bad mask from SAM cannot be repaired by a better inpainting model, and a good mask still produces a visible artifact if the inpainting backend has never seen similar texture.

The README states that any aspect ratio and 2K resolution are supported, which matters because many inpainting pipelines quietly resize inputs and lose detail. The README does not explain how that is achieved, so treat it as a claim to verify on your own images rather than a documented guarantee.

Installing Inpaint-Anything and running your first removal

The README requires python>=3.8 and gives three install commands. The first installs PyTorch, the second installs the vendored SAM package from the local segment_anything directory, and the third installs LaMa's requirements.

python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install -r lama/requirements.txt

On Windows the README recommends installing miniconda first, opening the Anaconda Powershell Prompt as administrator, and installing lama_requirements_windows.txt instead of lama/requirements.txt. That file sits at the repository root, so the Windows command differs from the third line above.

After the dependencies, the README says to download the model checkpoints. It does not print the checkpoint URLs in the portion of the README shown, so plan to find them in the repository's pretrained_models/ and weights/ directories or in the linked project pages. Nothing runs until the SAM and LaMa checkpoints are in place.

Once checkpoints exist, the entry points are plain scripts. The repository contains remove_anything.py at the top level, which is the Remove Anything driver, and sam_segment.py and lama_inpaint.py, which expose the two stages separately if you want to inspect the mask before inpainting. The local web UI lives in app/, and the README's news entry for 2023/4/24 states you can run the demo website locally. The README does not document the exact invocation flags for remove_anything.py, so read the script's argument parser before running it; do not assume flag names.

A reasonable first run is a single image with a single click point, using the same dog example the README shows in example/remove-anything/dog/. If the mask covers more than the object, the fix is at the SAM stage, not the inpainting stage.

Where the 2023 stack shows its age

The most concrete limitation is stated in the README itself: on the main branch, text-guided fill and replace use Stable Diffusion 2, which is marked as no longer downloadable. That means Fill Anything and Replace Anything, as described for main, may not be reproducible from scratch today. Remove Anything, which uses LaMa, is not affected by that note.

The second limitation is the beta branch's floor. main_2026 requires Python 3.12 or newer, PyTorch 2.7 or newer and CUDA 12.6 or newer, which the README attributes to SAM 3. That rules out older GPUs and older driver stacks. The README also states that the NeRF-based 3D path has not been end-to-end verified on that stack, so 3D removal on main_2026 is untested ground even by the maintainers' own description.

Third, the release history is thin. The only release listed is 0.1.0 from 2023-04-21, so there is no versioned upgrade path to follow; the project moves by branch and commit. The last push to the repository was on 2026-08-22, which is recent, but that activity is concentrated in the beta branch and in robotics work rather than in the main image pipeline.

Finally, the modality coverage is asymmetric. Video and 3D support removal only. If your job is inserting an object into a video or replacing a 3D object, this repository does not implement it, and the unchecked boxes in the feature list are the honest signal.

Inpaint-Anything versus a GUI inpainting extension

The alternative most people will weigh is an inpainting extension inside a Stable Diffusion web UI, where the mask is drawn by hand with a brush and the model is whatever checkpoint you have loaded. The difference is in where the mask comes from. In a web UI extension, the human paints the mask and the model only fills. In Inpaint-Anything, the mask is produced by SAM from a click, which makes the pipeline scriptable: you can run it over a folder without a person at the keyboard.

That trade goes both ways. A painted mask is often more precise than a prompted one, especially for thin structures such as wires, hair or a leash, where SAM's boundary can bleed. A web UI also gives you immediate visual feedback and lets you retry with a different sampler in seconds. Inpaint-Anything gives you reproducibility and batch throughput, but retrying means rerunning a script.

For text-guided work specifically, a web UI backed by a current Stable Diffusion checkpoint has an advantage over the main branch here, because main's SD 2 dependency is flagged as no longer downloadable. The main_2026 branch addresses that by moving to SDXL with optional FLUX.1-Fill, but it is labelled beta.

There is also a robotics-specific comparison the README makes directly. For Human-to-Robot pipelines such as Qwen-RobotManip and EgoEngine, main_2026's remove_hands.py erases human hands from egocentric video and exports the masks, and the README states it matches human annotation at IoU 0.96 on EgoMimic footage and reconstructs the background rather than blacking the arm out. That is a claim from the README, not an independent measurement, and it is scoped to that dataset.

main_2026: text prompts, ProPainter and hand removal

The beta branch changes enough that it deserves separate consideration. Segmentation moves from SAM 1 to SAM 3 with open-vocabulary text prompts, so --text_select "dog" finds every match instead of requiring a click per object. That single change is what makes unattended dataset processing possible, because a human no longer has to pick the object in each frame.

Video tracking moves from OSTrack to the SAM 3 video predictor, which the README notes removes one model and one checkpoint from the video path. Video inpainting moves from STTN to ProPainter. Text-guided editing moves from SD 2 to SDXL with optional FLUX.1-Fill. The robotics addition is remove_hands.py for batch hand removal.

The migration path the README gives is short:

git checkout main_2026
# then follow the Quick start in that branch's README

The important detail for anyone with an existing setup is the fallback claim: every legacy backend (SAM 1 or MobileSAM, OSTrack, STTN) is still selectable by flag, so you can fall back per stage rather than reverting the whole branch. The README does not list those flag names, so check the branch's own documentation before relying on that.

The README explicitly asks for contributions on the robotics side, naming more egocentric datasets, action retargeting, robot rendering and compositing, and newer inpainting backends. A project that advertises a beta branch and solicits PRs against it is signalling that the branch is not yet a stable target.

Licence, maintenance and what an upgrade actually costs

The repository is licensed Apache-2.0, which permits commercial use and modification provided the licence and notices are preserved. That covers the code in this repository. It does not automatically cover the model weights you download: SAM, LaMa, Stable Diffusion, SDXL, FLUX.1-Fill, ProPainter and the NeRF code each ship under their own terms, and the README links out to those projects rather than restating their licences. Check each checkpoint's licence separately before shipping anything; this is a description of the repository's licence file, not legal advice.

The upgrade cost is real and asymmetric. Staying on main means a frozen 2023 stack with a broken SD 2 download path and no versioned releases after 0.1.0, so the only upgrade mechanism is pulling commits. Moving to main_2026 means a Python 3.12, PyTorch 2.7 and CUDA 12.6 floor, new checkpoints for SAM 3, ProPainter and SDXL or FLUX, and an unverified 3D path. The per-stage flags are the mitigation, and they are the thing to test first: confirm that a legacy backend still runs under the new Python and CUDA versions before you migrate a working pipeline.

Because the last push was on 2026-08-22, the repository is not dormant. But activity on a beta branch is not the same as stability on the branch you depend on, and the README's own beta warning and its request for contributions should be read as the maintainers' assessment.

Editorial conclusion

Adopt Inpaint-Anything if you want the segmentation-plus-inpainting pipeline as readable Python you can call from your own scripts, and if a 2023-era SAM 1 and LaMa stack is acceptable. Do not adopt it if you need a Photoshop-style interactive editor, if you cannot download the SAM and LaMa checkpoints, or if you require the SD 2 text-guided path, which the README says is no longer downloadable. Before committing, verify that the checkpoints named in the README still resolve, run remove_anything.py on one of your own images, and check whether the main_2026 beta branch's Python 3.12, PyTorch 2.7 and CUDA 12.6 floor matches your GPU host.

Frequently asked questions

Official sources

1. geekyutao/Inpaint-Anything on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases