ffmpeg-quality-metrics: PSNR, SSIM, VIF, MSAD and VMAF from one command

One command compares an encode against its reference

ffmpeg-quality-metrics answers a narrow question: how far has this encoded copy drifted from the original. The tool takes two files, the distorted one and the reference one, and returns objective quality numbers for the pair.

The supported metrics are PSNR, SSIM, VIF, MSAD and VMAF, and the output has three shapes. You get per-frame metrics, global statistics covering minimum, maximum, average and standard deviation, and a graphical interface for looking at the results in a browser. JSON and CSV output are documented as well, so the numbers can be fed into a report rather than read off a terminal.

The tool is a Python package on PyPI, and the author is Werner Robitza. It is not a standalone encoder and it does not implement any metric itself. Everything heavy is delegated to FFmpeg, which means the tool inherits the behaviour of whatever FFmpeg build happens to be installed on the machine. That is the design, and it is also the first thing to check when a result looks strange.

The audience is engineers running comparisons in a pipeline: a codec experiment, a release gate, a regression check between two encoding settings. For a quick look at two arbitrary videos, the tool is more ceremony than the question deserves.

ffprobe checks the inputs and warns before the numbers appear

Before any metric is computed, both inputs are probed with ffprobe and compared on resolution, frame rate and duration. When they disagree, the tool prints a warning that spells out the consequence instead of quietly comparing different things. The documented examples read like this:

WARNING - Resolutions differ: distorted 854x480, reference 1280x720. The distorted video will be scaled to 1280x720 (bicubic) before comparison, which affects the metric values.
WARNING - Durations differ: distorted 3.000 s, reference 2.000 s. Only the first 2.000 s are compared; the rest of the distorted video is ignored.

That is a genuinely useful design choice for a metric tool, because a PSNR computed after a bicubic upscale is a different measurement from one computed at native resolution. A frame rate mismatch is checked the same way, and the scaling in the simple case is automatic: the distorted file is scaled to the resolution of the reference, and the default metrics are PSNR and SSIM.

Two failure modes are harder. The tool stops with an error if an input cannot be read or has no video stream at all. And the checks themselves are optional in a specific sense: ffprobe is expected next to the ffmpeg executable or on the PATH, and if it cannot be found the input checks are skipped rather than failing. A run without ffprobe therefore produces numbers without the warnings that would have told you the comparison was uneven.

--dist-delay trims a lead-in offset without touching the files

Timing differences between a reference and an encode are common and they quietly corrupt the comparison, so the tool has a dedicated option for them.

The --dist-delay flag trims the unmatched lead-in before the comparison starts. A positive value means the distorted file starts that many seconds later than the reference, so the reference lead-in is cut. A negative value means the distorted file starts earlier, so its lead-in is cut instead. The documented example uses a value of 12.5 seconds:

ffmpeg-quality-metrics distorted.mp4 reference.mp4 \
  --metrics vmaf --dist-delay 12.5

The implementation detail that matters is in how this is done. The files are decoded and trimmed inside the filtergraph, and neither input is re-encoded. Metrics then stop at the end of the shorter aligned stream, so trailing content in the longer file is not compared against a repeated final frame. Avoiding a re-encode matters for measurement, because a second generation of compression would contaminate the very quantity being measured.

The flag also lets you ask for one metric explicitly, as the example does with --metrics vmaf. Without it, the default pair of PSNR and SSIM is computed.

Raw YUV is refused, Y4M and FFV1 are the accepted formats

There is one hard limitation that catches people out, and the documentation is unusually blunt about it: raw YUV files cannot be read with this tool. The reasoning given is that everyone should be using lossless containers like Y4M or FFV1.

That is a defensible position rather than a bug, because a raw YUV file carries no frame rate, no resolution metadata and no notion of its own boundaries, and guessing any of those silently is exactly the kind of error that makes a metric meaningless. The fix is a conversion step, and the readme gives the command:

ffmpeg -framerate 24 -video_size 1920x1080 -pix_fmt yuv420p -i input.yuv output.y4m

You have to supply the frame rate, the frame size and the pixel format yourself, because that information is exactly what the raw file does not contain. The documentation notes that the options should be adjusted as needed.

This is the main friction point for anyone measuring camera output rather than encoder output. If your pipeline starts from raw dumps, a conversion has to happen before any comparison, and getting the framerate wrong in that conversion is the failure mode to guard against, since every number downstream inherits it.

Five metrics, three output shapes and an optional dashboard

The metric list is short and the choices are conventional. PSNR is the peak signal to noise ratio in decibels, where higher is better, and it comes with per-channel components such as mse_avg, mse_y, mse_u, mse_v and the matching psnr values for average, luma and the two chroma channels. SSIM, VIF, MSAD and VMAF sit alongside it, each with its own scale and submetric breakdown, and the readme's table marks which of them are calculated by default.

Everything else follows from that. Per-frame values let you see where the damage happened rather than only how much of it there was, and the global statistics give the min, max, average and standard deviation that a threshold check needs.

The browser interface is an optional extra rather than part of the core. The packaging declares it as an extra named gui, with plotly and dash as its dependencies, and it installs a second entry point, ffmpeg-quality-metrics-gui, next to the main ffmpeg-quality-metrics command. So the plot is a separate command that reads results the metric run produced, not a mode flag on the same one.

A convenience worth noticing is that the tool needs no configuration file, no database and no service. It is a single executable with a single file argument pair, which makes it easy to slot into a script that already shells out to FFmpeg.

Three install routes, plus a Docker image that bundles its own FFmpeg

Installation is deliberately shallow. The shortest route runs the tool without installing anything at all:

uvx ffmpeg-quality-metrics

The two persistent routes are pipx, which installs the command into its own environment, and plain pip:

pipx install ffmpeg-quality-metrics

pip3 install --user ffmpeg-quality-metrics

What none of these bring is FFmpeg itself, and that is the real requirement list. The documentation asks for Linux, macOS or Windows, Python 3.10 or higher, and FFmpeg 7.1 or higher, with the ffmpeg executable on the PATH and ffprobe next to it. VMAF needs more: the FFmpeg build should include libvmaf, the VMAF model files are bundled with the package or can be downloaded from the VMAF project, and the newer VMAF v1 models require an FFmpeg build linked against libvmaf 3.2.1 or newer.

The container image removes that friction. Its Dockerfile starts from python:3.13-slim, downloads an FFmpeg build from the BtbN builds repository at version 9.0, copies ffmpeg and ffprobe into the image and installs the package, ending with a single entry point:

ENTRYPOINT ["ffmpeg-quality-metrics"]

The comment in that Dockerfile notes the reason for the particular build: BtbN builds include libvmaf 3.2.1, so VMAF works out of the box in the container.

The executable was renamed, and the version numbers move quickly

One migration note is worth repeating because it breaks scripts. Earlier versions installed an executable called ffmpeg_quality_metrics with underscores. To harmonise it with other tools, the executable is now called ffmpeg-quality-metrics with a hyphen, and the documentation asks you to remove the old file, suggesting you run which ffmpeg_quality_metrics and delete what it points at.

The release cadence suggests an active project. The recent versions are v3.13.0 on 2026-10-14, v3.12.7 and v3.12.6 on 2026-10-11, and the last push was on 2026-10-14 with the repository not archived. Patch-level releases landing within hours of each other are the signature of a tool that tracks FFmpeg itself, since a new FFmpeg release can change filter behaviour that the metric numbers depend on.

The licence is MIT, with a LICENSE.md file at the root and the same identifier in the package metadata, so embedding the tool in a commercial pipeline carries no copyleft obligation. The project metadata also lists it as a beta, with classifiers from Development Status :: 4 - Beta, and the repository includes a CHANGELOG.md, DEVELOPERS.md and a tests directory. The development dependencies are pytest, ruff, pre-commit and pdoc, and the package builds with the uv_build backend, which is a modern and fairly opinionated choice that ties the release process to that tool chain.

Editorial conclusion

Use ffmpeg-quality-metrics when you need a repeatable number for how far an encode drifted from a master, especially in a pipeline where the reference and the distorted file are produced by the same script. Do not reach for it to compare two different sources, since the tool aligns one file to the other and scales it, and it refuses raw YUV, which rules out quick checks on camera dumps without a conversion step. Verify first that the FFmpeg build on your machine is 7.1 or newer, that it is linked against libvmaf if you intend to compute VMAF, that ffprobe sits next to the ffmpeg binary so the input checks run, and that your references are in a lossless container such as Y4M or FFV1.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. slhck/ffmpeg-quality-metrics on GitHub