# immich-deduper: finding near-duplicate photos in an Immich library

> immich-deduper reads your Immich PostgreSQL database, runs every image through ResNet152, stores the feature vectors in Qdrant and groups the near matches so you can decide what to keep. Deletions go to Immich's own trash, and the tool refuses to do anything until you turn that feature on.

**RazgrizHsu/immich-deduper** — duplicate photo finder for Immich - find and remove similar images

- Repository: https://github.com/RazgrizHsu/immich-deduper
- Stars: 675 · Forks: 20
- Language: Python
- License: GPL-3.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/razgrizhsu-immich-deduper

## It reads Immich's database directly, and it needs the trash feature on

immich-deduper is a duplicate photo finder and remover for Immich, previously called Immich MediaKit, and it finds similar images using deep learning. The architecture is stated plainly in the README's how it works section. It fetches users and assets data from the Immich PostgreSQL database, processes images through ResNet152 to extract feature vectors, stores those vectors in the Qdrant vector database, uses vector similarity to identify similar or duplicate photos, displays groups based on the configured threshold, and manages asset deletion by updating the Immich database directly. The deletion part follows Immich's own deletion logic for compatibility, and the README puts an important note next to it: you have to enable the trash feature in Immich settings first. Deleted assets then appear in Immich's trash, where you can permanently delete them or restore them. That ordering is deliberate, and it is the single most important operational detail in the project.

## The pipeline runs as a container with Qdrant beside it

The project ships as a Python application with a Dockerfile that has a build argument for device selection, defaulting to cpu. The base image is python:3.12-slim-bookworm, and the requirements file is copied in and installed based on the device, with separate requirement files for cpu, cuda and cuda128, and an explicit error for an unsupported device value.

```dockerfile
FROM python:3.12-slim-bookworm

WORKDIR /app

ARG DEVICE
ARG DEDUP_PORT=8086
ENV PORT=${DEDUP_PORT}
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    TORCHINDUCTOR_CACHE_DIR=/tmp/torch_inductor
```

The image installs libimage-exiftool-perl through apt, exposes the port, and defines a health check that hits the application over HTTP with a short timeout. The command at the end runs the app as a module. The port and the Qdrant connection are configured through environment variables, and the example file shows the full set.

```bash
DEDUP_DATA=./data
DEDUP_PORT=8086

# qdrant ports
QDRANT_PORT=6333
QDRANT_URL=http://localhost:6333

# immich
IMMICH_PATH=/path/to/your/immich/photos
```

The local requirements file pins torch at about 2.11 and torchvision at about 0.26, with libsass, and the pyproject configures pyright to check the src directory with Python 3.12 as the target version.

## Similarity is a threshold, and where you set it decides what you see

The flexible threshold is the setting that decides what counts as a duplicate, and the README gives the range. Exact duplicates start at 0.97 and above, while similar shots appear from 0.60 and above. Below that you are looking at shots that share composition or subject rather than the same frame, which is where review effort explodes. The threshold is used to select the groups shown, so a lower value means more groups and more decisions. Two related settings shape the same output. Path Filter only shows groups where at least one asset's path contains the filter pattern, which is how you focus on one folder or one external library, and groups that do not match are auto-resolved, which the README notes means you need Reset Records to search them again. Exclude Settings skips files by extension or by filename pattern, with the documented example of a drone photographer who shoots RAW and JPEG at the same time and wants to keep both without them being flagged.

## Kept photos are protected when they show up in a later group

The Kept Lock feature is the part that makes a review queue usable. Photos kept by any action are marked Kept, and when a Kept photo appears in a later group it is shown as a reference only: it cannot be selected, and Auto Selection and all the actions skip it. Without that, a photo that spans three groups would be a candidate for deletion three times, and the most likely outcome is that you delete the one you wanted. Two commands manage the marks. Clear records and Keep resolved clears out search records that have not been resolved yet while keeping the ones already marked as resolved. Reset records resets all search records including the resolved ones. The distinction between those two is what lets you run a search again over a subset without losing the progress you made on the rest.

## Auto Selection picks a keeper, but the deletion is still yours

Auto Selection chooses which photos to keep or delete automatically after you run Find Similar, picking the best photo by date, size, EXIF and other criteria. The README is clear about the workflow: enable auto selection, scroll through to review, then hit one of the four action buttons at the top. So the tool does the ranking and you do the acting, which is the right division of labour for something irreversible. The Find Similar action starts searching for the next photo matching your Threshold Min settings and shows it in the current tab, and the Auto Find Next option controls what happens after you resolve or delete a group: with it enabled, resolving the current group triggers a search for the next unprocessed photo, and with it disabled, the system switches to the pending tab so you can work through everything found before searching again.

## The local copy is a snapshot, and the README says so plainly

The sync behavior section is the one to read before you trust the tool with a large library. The deduper does not auto-sync with Immich. The local database is a snapshot from your last Fetch, and any change you make on the Immich side leaves the two sides drifting apart until you open the Fetch page and run Fetch: Get Assets again for the affected user. Re-running Fetch reconciles the local database with Immich's current state, and newly brought-in assets still need a pass on the Vector page before they appear in similarity search. That two-step requirement is easy to miss, and it means the first time you use the tool after importing new photos, the search results will be incomplete until you have done both.

## Related Tree, cross-user detection, and the metadata merge in beta

Two features extend the basic group model. Cross-user detection finds duplicates across multiple Immich users, with the scope depending on how you import: importing a single user detects duplicates within that user only, while importing multiple users detects them across all of them. That distinction matters for shared family libraries. Related Tree discovers connected similarities, that is, photos related to photos that are themselves related, which finds clusters that a flat similarity search would split apart. The metadata merge feature is marked as beta and transfers metadata from deleted photos to the ones you keep, covering albums, favorites, tags, rating, description and location. It is the feature most likely to lose data in a way you would not notice, since a copied album membership is invisible after the fact, and it is the one to test on a small library first.

## Conclusion

immich-deduper fits an Immich user with a library full of burst shots, screenshots and re-imported folders who wants a review queue rather than a blind cleanup, and who is comfortable pointing a tool at their database and photo path. It does not fit anyone who wants one-click automatic deletion, because every group needs a human decision, and it does not fit a deployment where direct database writes are off the table. Verify first that the trash feature is enabled in Immich, run a small import before your whole library, and treat the Fetch page as the source of truth since the local copy is a snapshot that drifts out of date.

## FAQ

### How does immich-deduper find duplicate photos?

It fetches users and assets from the Immich PostgreSQL database, processes images through ResNet152 to extract feature vectors, stores the vectors in the Qdrant vector database, and uses vector similarity to group similar or duplicate photos based on the configured threshold.

### What do I need to set up before running immich-deduper?

The environment variables include the port, the Qdrant URL and port, the path to your Immich photos, and the PostgreSQL host, port, database, user and password. The README also insists you enable the trash feature in Immich settings first, so that deleted assets land in Immich's trash and stay recoverable.

### Does immich-deduper sync automatically with Immich?

No. The local database is a snapshot from your last Fetch, and changes made in Immich leave the two sides drifting apart until you run Fetch: Get Assets again for the affected user. Newly fetched assets also need a pass on the Vector page before they appear in similarity search.

### How do I avoid deleting a photo I wanted twice?

Use the Kept Lock behaviour. Photos kept by any action are marked Kept, and when such a photo appears in a later group it is shown as a reference only, cannot be selected, and is skipped by Auto Selection and all the actions.

## Sources

- [Issues](https://github.com/RazgrizHsu/immich-deduper/issues)
- [License: GPL-3.0](https://github.com/RazgrizHsu/immich-deduper/blob/main/LICENSE)
- [RazgrizHsu/immich-deduper on GitHub](https://github.com/RazgrizHsu/immich-deduper)
- [README](https://github.com/RazgrizHsu/immich-deduper/blob/main/README.md)
- [Releases](https://github.com/RazgrizHsu/immich-deduper/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/razgrizhsu-immich-deduper