Video Duplicate Finder: similarity-based duplicate detection for video files

What Video Duplicate Finder actually solves

Filename-based duplicate finders are useless on a video library. The same movie exists as a 1080p rip, a 720p re-encode, a version with a subtitle burn-in and a version with a channel watermark, and none of those copies share a name, a size or a byte. Video Duplicate Finder is built for that situation. Its stated purpose is to find duplicated video and image files based on similarity, and the README calls out the cases other duplicate finders miss: different resolution, different frame rate, watermarked copies.

The audience is anyone with a large local media collection: people consolidating drives, archiving family footage, or cleaning up a NAS. The project ships four front ends for that audience. A desktop GUI for Windows, Linux and macOS, a headless CLI for scripting, a self-contained web server for remote or NAS use, and a Docker image. That spread is the point: the same scanning engine is reachable from a desktop session or from a container with no display at all.

How the matching pipeline is layered

The core scan is visual and is described as authoritative. Everything else is additive. The classic comparison produces the baseline duplicate list, and the optional passes can only add pairs, never remove them. The README states this explicitly for AI matching: the AI pass only adds pairs it is confident about, so enabling it never hides results you would otherwise get. That is a deliberate design choice and a good one, because it means a false negative in an optional model cannot cost you a duplicate the classic scan already found.

Partial clip detection runs as an optional second phase after the normal visual scan. It uses an audio fingerprinting pipeline, described as Chromaprint-style chroma extraction plus sliding-window Hamming similarity matching. Candidates come from audio, and by default each audio match is then visually confirmed by comparing frames at the matched offset. Matched pairs get a Clip Offset column showing where in the source the clip starts.

AI matching is a separate, independent switch. It uses neural image embeddings from a DINOv2 vision model running through ONNX Runtime. Two capabilities come from it: transformed copies such as cropped, mirrored, zoomed, letterboxed or color-graded versions, marked with an AI chip in results, and visual partial detection, which matches sampled keyframes with a consistent time offset. The visual variant matters because it works on silent, muted and re-dubbed videos, where the audio fingerprinting pass has nothing to chew on.

Installing Video Duplicate Finder and running a first scan

There is no package manager install. You download a build from the releases page, and the packages are named per platform and per front end: GUI-<platform>, CLI-<platform> and Web-<platform>. The README points at the 4.1.x daily build, whose attachments are rebuilt and replaced on every commit, and notes that the final classic-UI build stays on the 4.0.x release.

On Linux, the GUI package is a binary you mark executable and run. FFmpeg and FFprobe are required, and on first launch VDF attempts to download them automatically, but installing them yourself is the predictable path:

sudo apt-get update && sudo apt-get install ffmpeg
chmod +x VDF.GUI
./VDF.GUI

On Windows the README says to take the latest FFmpeg GPL shared package from ffmpeg.org, extract ffmpeg.exe and ffprobe.exe into the same folder as VDF.GUI.exe, into a subfolder named bin, or onto your PATH. One constraint is easy to miss: the native FFmpeg binding requires FFmpeg 8.x shared libraries, not the master branch. If you have a distro build that is older, the native path is not what you are getting.

For a headless host, the repository ships a Docker Compose file that runs the web server on port 8080 and mounts your media read-only:

services:
  vdf-web:
    image: ghcr.io/0x90d/vdf-web:latest
    ports:
      - "8080:8080"
    volumes:
      - /mnt/nas/movies:/mnt/nas/movies:ro
      - vdf-db:/root/.config/VDF
      - vdf-state:/root/.local/state/VDF

The two named volumes are not decorative. vdf-db holds web-settings.json and web-credentials.json, vdf-state holds ScannedFiles.db. Drop them and you lose your settings and your scan database. Authentication is on by default: the compose file comments show VDF_WEB_PASSWORD to set a password, otherwise one is auto-generated and you have to check the container logs, and VDF_WEB_AUTH=false to disable authentication entirely. The two VDF_TRUSTED_PROXIES and VDF_TRUSTED_PROXY_NETWORKS variables only matter behind an HTTPS reverse proxy, where they let the auth cookie be marked Secure.

For automation, the CLI exposes the AI switches as flags: --ai-matching, --ai-percent, --ai-partial and --ai-partial-hit-percent.

Where the optional passes cost you

The AI pass is off by default, and the reason is footprint. On first use VDF downloads two components, roughly 100 MB once: the ONNX Runtime library from Microsoft's official release and the embedding model, integrity-checked against a pinned SHA256. They are stored next to the scan database. The GUI asks before downloading. CLI, Web and Docker download automatically once an AI option is enabled, which is exactly the wrong behaviour on a metered or air-gapped machine unless you plan for it.

Runtime cost is stated as roughly 50 ms per file during hashing, with embeddings cached in the scan database at about 2 KB per file, so rescans stay fast. Visual partial detection keeps its keyframe cache in a separate DenseEmbeddings.db sidecar at about 25 KB per video, and the README says that sidecar cleans itself up. On a library of tens of thousands of files, 2 KB per file is real but manageable; 25 KB per video for the dense pass is the number to check against your disk budget before you turn it on across everything.

Partial clip detection has a harder limit: it requires audio tracks in both files. Videos without audio are skipped entirely, and the README redirects you to the visual variant under AI matching for those. So the two clip-detection mechanisms are not redundant, and on a silent-footage library the audio one contributes nothing.

The thresholds are where the tuning lives, and the defaults are conservative. Minimum clip-to-source ratio is 10 percent, minimum audio similarity 80 percent, minimum visual similarity 85 percent with visual confirmation on. For AI, the embedding threshold defaults to 94 percent, with the README suggesting you lower it to about 92 to find more aggressively edited copies at slightly higher false-positive risk. For visual partial detection, the AI frame hit threshold defaults to 89 percent, and at least 4 hits must agree on one time offset before two videos are paired; the README says to raise it if unrelated videos get paired. That last sentence is the honest failure mode: aggressive settings pair unrelated videos.

Video Duplicate Finder versus Czkawka and similar finders

Czkawka is the comparison people search for, and the difference is scope rather than quality. Czkawka is a general-purpose duplicate and junk file finder: it covers many file categories and finds duplicates by content hashing, which is exact-match logic. Two video files that are the same content at different resolutions or frame rates are not duplicates to a hasher, because their bytes differ.

Video Duplicate Finder inverts that. Its baseline is similarity on video content, and its optional layers exist precisely to catch the pairs hashing cannot see: audio-fingerprint matching for a clip ripped out of a longer recording, and neural embeddings for a cropped, mirrored or color-graded copy. If your problem is photos, documents and archives, a general finder is the better tool. If your problem is a media library where the same thing exists in six encodings, similarity matching is the only approach that produces a useful list. The trade-off is that similarity means thresholds, and thresholds mean tuning and false positives, which an exact hasher never gives you.

Upgrades, the 4.1 interface and the 3.x migration

Version 4.1 introduces a redesigned interface, and the project keeps the final classic-UI build on the 4.0.x release for people who prefer it. Databases and settings are stated to be compatible both ways between 4.0.x and 4.1, so moving between those two is not a data decision.

The 3.x upgrade is the one to think about. The scan database is migrated automatically on first load. Cached image hashes are recomputed on the next scan because image processing moved from ImageSharp to FFmpeg; video hashes are unaffected. The README states that downgrading back to 3.x after the migration is not recommended, and keeps the last 3.x build on the 3.0.x release. Practically, that means an upgrade to 4.x is a one-way door for your database, so copy the database file before the first 4.x launch if you might want to go back.

The repository ships no licence file at the top level, and the README does not state a licence. That is worth resolving before you ship the CLI or the web server inside a product, because the FFmpeg dependency the README points at is the GPL shared build. Treat the licence question as open and check the repository yourself rather than assuming.

Maintenance is easy to read from the release tags: 4.1.x on 2026-07-07, 4.0.x on 2026-06-11, and an ai-models-v1 asset release on 2026-07-12. The repository is not archived, and the last push was on 2026-09-21.

Editorial conclusion

Adopt Video Duplicate Finder if your duplicates are videos that were re-encoded, resized, re-framed or clipped, and you want that matching to stay on your own machine. Do not adopt it if you need a plain file-level deduplicator for mixed document types, or if you cannot install FFmpeg and FFprobe on the host. Before committing, verify three things on your own media: that the FFmpeg version you have is the 8.x shared build the native binding wants, that you are willing to keep roughly 2 KB per file of embeddings in the scan database if you turn AI matching on, and that you can live with the 4.1 interface, since the classic UI is frozen on the 4.0.x release. Databases and settings are stated to be compatible between 4.0.x and 4.1, so the interface choice is reversible in one direction only: downgrading after a 3.x database migration is not recommended.

Frequently asked questions

Official sources

1. 0x90d/videoduplicatefinder on GitHub
2. Issues
3. README
4. Releases