ffmpeg-sidecar hands you FFmpeg as an iterator over raw RGB frames

Video as an array of frames

The core goal of this project is to provide a method of interacting with any video as if it were an array of raw RGB frames.

The documentation is honest about the premise. That is what video fundamentally is, but there is a whole pandora's box of complexity in terms of receiving and decoding video before you get there.

So the problem is not the idea of frames, it is the plumbing. Every format, every container, every codec, every pixel format conversion, every rotation metadata field, every stream mapping is a detail that sits between an application and the pixels it wants. A library that gives you an iterator of frames has absorbed all of that.

Using FFmpeg as the core engine provides interoperability between a massive range of formats, containers, extensions, protocols, encoders, decoders, hardware accelerations and more. That is the argument for the design: the hard part of video handling has already been solved once, thoroughly, by FFmpeg, and a wrapper can inherit that for free.

The crate is MIT licensed, the author is Nathan Babcock, the version is 2.7.0, the edition is 2021, and the minimum Rust version is 1.88. The release history shows steady work: v2.7.0 on 2026-10-18, v2.6.0 on 2026-09-30 which was the release for FFmpeg 9, and v2.5.0 on 2026-04-02 which was the release that added customising the downloaded binaries.

Why wrap the CLI instead of linking the library

The documentation explains the alternative and its drawbacks before making the case.

One method of using FFmpeg is low-level bindings to the code used inside the CLI itself, and there are good crates in the Rust ecosystem that do this. The drawbacks are listed: a difficult, time-consuming build, toolchain and dependencies situation, especially on Windows. Complexity, especially for beginners. And you end up manually re-implementing a lot of the standard conversions you need from scratch.

That third point is the one that matters most in practice. With bindings you get the ability to call the library, not the CLI's carefully assembled argument lists. The CLI already knows how to pick a pix_fmt, a scale filter and a codec for a given output, and reproducing that logic is work you would otherwise be doing for every command.

By wrapping the CLI the crate avoids those downsides and also solves some of the pain points of using the CLI on its own. The first is that raw data can easily move in and out of FFmpeg instances, or pipe between them, because under the hood they are moving across stdin and stdout.

The second is richer semantic information recovered from the FFmpeg stderr logs, including progress updates such as frame number, timestamp, speed and bitrate, input and output metadata and stream mappings, and warnings and errors. That last one matters: an FFmpeg process that fails prints the reason to stderr, and a wrapper that surfaces it turns a silent failure into a readable error.

The builder API, and the frame iterator

The hello world example shows the whole shape of the interface.

use ffmpeg_sidecar::command::FfmpegCommand;

fn main() -> anyhow::Result<()> {
  // Run an FFmpeg command that generates a test video
  let iter = FfmpegCommand::new() // <- Builder API like `std::process::Command`
    .testsrc()  // <- Discoverable aliases for FFmpeg args
    .rawvideo() // <- Convenient argument presets
    .spawn()?   // <- Ordinary `std::process::Child`
    .iter()?;   // <- Blocking iterator over logs and output

  // Use a regular "for" loop to read decoded video data
  for frame in iter.filter_frames() {
    println!("frame: {}x{}", frame.width, frame.height);
  }

  Ok(())
}

Four things are happening there. The builder is compared to std::process::Command, so if you have used that API the shape is familiar. The presets are argument aliases with discoverable names, so testsrc and rawvideo replace the corresponding FFmpeg flags and your editor can complete them. Spawn returns an ordinary std::process::Child, which means the process is a real child process you can manage. And iter turns that child into a blocking iterator over logs and output.

The frame loop then uses filter_frames to pick out the decoded frames from that stream, and each frame carries its width, its height and its data as raw RGB pixels.

The argument aliases are the discoverability argument made concrete. The documentation lists argument presets and aliases with discoverable names through Intellisense and autocomplete as one of the advantages of the CLI approach, and that is exactly what a named method on the builder gives you that a string of flags does not.

The binary, and downloading it at runtime

The remaining downside of the CLI approach is stated plainly: the size of the FFmpeg binary itself, but it is less than 100MB when zipped. It can be automatically downloaded by the crate, so you may choose to not even ship it with your own application and instead download it at runtime.

That choice is the interesting design decision, and it splits two different audiences. For a developer working locally, the download is a convenience. For an application shipped to end users, downloading a codec binary on first run is either an acceptable cost of a smaller installer or a non-starter behind a firewall.

The download call is a single function:

ffmpeg_sidecar::download::auto_download().unwrap();

You can do this once to set up a dev environment, or include it as a feature of your client application. The download covers Windows, MacOS and Linux, and there is a companion auto_download_with_progress function for the case where you want to show the user something.

Two details are worth knowing. The KEEP_ONLY_FFMPEG environment variable, set to 1 or true, skips installing ffplay and ffprobe during autodownload, which is useful if you only need the encoder. And if you are handling extraction yourself, unpack_ffmpeg_without_extras is the entry point.

The feature flags in the manifest control this. The default feature set is download_ffmpeg, and it pulls in ureq for the HTTP request plus tar, xz2 and zip for extraction, with the archive crates selected per platform: tar and xz2 on Linux, zip elsewhere, with deflate enabled. The named_pipes feature is separate, bringing in winapi on Windows and nix on Unix.

The examples show the patterns people actually need

The example set is a better guide to the library's scope than the feature list.

There is hello_world, which reads raw video frames, and h265_transcode, which decodes H265, modifies the decoded frames and writes back to H265. That second one is the shape of a real use case: treat the video as frames, change them, encode them again.

There is ffplay_preview, which pipes an FFmpeg instance to FFplay for debugging purposes, and named_pipes, which pipes multiple outputs from FFmpeg into a Rust program, gated behind the named_pipes feature. There are also examples for ffprobe, for metadata, for a progress listener, for sockets, for a terminal video preview, for a mic meter, for a game of life that renders to a video, for a trigger, and for whisper.

That last one is worth pausing on. Using FFmpeg to capture audio and hand it to a speech model is a pattern people build constantly, and having it as an example in the repository says more about the intended audience than a feature list does.

Each example is run with cargo run and the example name, and the named pipes one needs the feature flag, cargo run --example named_pipes --features named_pipes. The documentation also points at the unit tests in src/test.rs as a further source of use cases, and says to check the examples for many other uses.

For async Rust or tokio, the documentation points at a separate crate, async-ffmpeg-sidecar, rather than trying to be both.

Version support, and the parts borrowed from elsewhere

Two housekeeping details are worth recording.

The first is the FFmpeg version coupling. The v2.6.0 release was labelled FFmpeg 9, so the crate tracks the major version of the FFmpeg binaries it downloads, and the auto-download function gets you a matching build rather than whatever your system has installed. The v2.5.0 release added the ability to customise the downloaded binaries, with an examples/download_ffmpeg.rs file showing how, which is the hook for anyone who needs a build with specific codecs or hardware acceleration enabled.

The second is the lineage. The project says it was inspired loosely by the Node.js fluent-ffmpeg package, which does something similar in JavaScript, and that it uses the setup-ffmpeg project for GitHub Actions and as a reference for the auto-download behaviour.

The repository layout is consistent with a crate that ships examples as first class documentation. Alongside the source there are examples, a guides directory, a scripts directory, a rust-toolchain file, a rustfmt.toml, an output directory and a .vscode directory, and the CI workflow badge is in the readme.

The manifest is deliberately lean. The dependencies are anyhow for error handling and ureq for the download, with the default feature pulling in the archive crates. The docs.rs metadata sets all-features and passes a docsrs cfg, so the published documentation shows the named_pipes API as well as the default one.

Editorial conclusion

Adopt ffmpeg-sidecar if you want FFmpeg's format coverage in Rust without the build pain of low level bindings, especially if you are on Windows or you want frames as raw data. Do not adopt it if you need a small static binary, since the FFmpeg binary itself is under 100MB zipped and can be downloaded at runtime instead of shipped. Before you start, check three things: whether your application can tolerate a subprocess rather than an in-process library, since the crate is a wrapper around the CLI and the data moves over stdin and stdout, whether you want the automatic download behaviour, which is on by default through the download_ffmpeg feature, and whether you are in an async context, where the documentation points you at a separate async crate instead.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. nathanbabcock/ffmpeg-sidecar on GitHub
4. README
5. Releases