dash-mpd-cli: downloading DASH streams from the command line

What it is, and the protocol it speaks

dash-mpd-cli is a command line application for downloading media content from a DASH MPD file, as used for on-demand replay of television content and video streaming services.

The readme spends its first paragraph on the protocol, and that is the right way to present a tool like this, because everything the tool does follows from the shape of the manifest.

DASH, also called MPEG-DASH, is a technology for media streaming over the web, commonly used for video on demand and for replay services. The Media Presentation Description is an XML document that lists the resources, the manifest or playlist, forming a streaming service. A DASH client uses the manifest to determine which assets to request in order to perform adaptive streaming of the content.

So the manifest is the index, and it is an index of alternatives rather than a single file. It describes the content in a set of periods, and within each period a set of representations, each with its own bitrate, codec, resolution and possibly its own language. A client picks one and adapts. The readme notes that the manifests work with content using a long list of codecs, including H264, HEVC, AV1, AAC, VP9, MP4A and MP3, and containers including MP4, WebM, Matroska and AVI, and it links to a site with a good explanation of adaptive bitrate streaming for anybody who has not met the model before.

The application itself, then, does the downloading: it selects the alternative with the most appropriate encoding in terms of bitrate and codec, fetches segments of the content using HTTP or HTTPS requests, and muxes audio and video segments together. There is also support for downloading subtitles in several formats.

And the scope limit is stated without hedging: the tool works for streaming that uses DASH adaptive streaming, and it does not currently have support for HLS streaming with its own manifest format. That is worth knowing before you download it, because half the streaming services out there use the other protocol.

The project is a Rust application published to the package registry, with badges for the build, the dependency status, the container image size and the download count, and it builds on a separate library of its own that does the manifest parsing. There is a user manual hosted alongside it, and a terminal recording in the readme, which is the fastest way to see what the output looks like.

Segments, periods, and containers

The feature list is long, and two groups of items in it are worth separating because they are where the hard work is.

The first is segment addressing. The readme states support for all forms of it: the base with a byte range, the timeline form, the template with a duration, the template with an index, and the explicit list. These are the five ways a DASH manifest can say where the pieces of a stream live, and they are not variations on a theme. The timeline form is the one most services use, because it expresses irregular segment boundaries, and it is the one where a client has to do the arithmetic. The index range form describes a single file that contains many segments, and the list form names them. A tool that supports only one of them works with one kind of service, and the readme's claim of all five is the difference between a general downloader and one that happens to work for the site you tried it on.

The second is containers and codecs. The readme says the tool supports media containers of the types supported by mkvmerge, ffmpeg, VLC or MP4Box, which covers the common broadcast and streaming formats, and all the codecs those applications support. And it says the tool can handle any video resolution available on the streaming server, including 1080p and 4K content.

That last point is really about the muxing step. Once you have fetched the segments, the audio and the video have to be put into one file, and that means running an external tool. Which tool can be chosen by what the segments are: the tool that understands broadcast containers is not the one that understands a WebM stream, and the readme's framing, that the supported containers are the ones supported by these four applications, is a way of saying that the muxing is delegated.

Then there is the multi-period handling, which is the subtlest of the three. When a service splits its content into periods, which is what a live or long-running service does so that it can update one part of the stream without republishing the whole thing, the segments from different periods may not be compatible with each other. The readme says the media from different streams is saved in a single container if the formats are compatible, in terms of resolution, codecs and bitrate, and otherwise in separate containers, and there is a command line option to turn the concatenation off entirely.

So the tool does not blindly concatenate. It checks, and it degrades to separate files when the check fails, and it lets you force the separate behaviour. That is a careful design choice for a case where the wrong answer is a file that plays the first minute and then breaks.

Choosing what you get

The selection criteria are listed in one long sentence, and every item in it is a thing a viewer would want to specify.

You can select the streams to download based on the audio language, the quality, the horizontal and the vertical resolution of the video stream, the video codec, the stream id, and the role annotation.

That last one deserves a note, because it is the least obvious. A role annotation is a flag on a representation that says what it is for, and the readme gives the example of a main role instead of an alternate role. A service that publishes a main audio track and alternate ones, which is what a service with multiple audio descriptions or multiple commentary tracks does, marks them in the manifest, and the tool can pick the main one. Without that, you would be choosing the first track with the right language, which is not the same thing when a service publishes a description track alongside the dialogue.

The language criterion matters for the same reason. A service that publishes the same programme in several languages, or a film with the original and the dubbed version, has several audio representations, and language is how you choose between them. And the resolution criteria are split into horizontal and vertical, which is a small thing that tells you the author has thought about portrait video, where the height is the number that matters.

The quality criterion is the one that needs the most care, and it is worth understanding what it means in this context. The manifest describes a ladder of representations, and a client picks one based on the bandwidth it thinks it has. A downloader has no bandwidth to think about, so it has to be told. The readme's wording, that the application selects the alternative with the most appropriate encoding in terms of bitrate and codec, is the answer: the tool is given a quality and it takes the representation closest to it.

That is a different model from the streaming client, and it is the right one for a download. A streaming client picks what it can play smoothly; a downloader picks what you asked for, and then the bandwidth question becomes the throttle argument instead.

The network side, which is where real services differ

The network features are the ones that a user hits when the simple case does not work, and the readme covers them thoroughly.

Content can be fetched over HTTP, HTTPS and HTTP/2, and the bandwidth can be throttled with a rate limit argument, which is the difference between downloading a forty gigabyte 4K stream over a hotel connection and downloading it overnight.

Proxies are supported for SOCKS and for HTTP, through a command line argument, and the readme also lists the environment variables that work at a system level: the upper and lower case forms of the HTTP, HTTPS and all-proxy variables, each for the connection types they imply, plus a flag that disables the system proxy entirely. The list of environment variables is the conventional set, so a machine that already exports them for other tools will behave the way you expect.

Authentication covers HTTP Basic and Bearer tokens, with separate arguments for the username and password and for the token. And the detail that matters is in the next sentence: the authentication information is sent both to the server hosting the manifest and to the server hosting the media segments, the latter often being a content delivery network.

That last point is a real operational fact about streaming services, and getting it wrong is a common failure. The manifest and the segments are frequently served from different hosts, and a tool that sends the credentials only to the manifest host gets a manifest and then a wall of unauthorised requests for the segments. Sending them to both is what makes the tool work against a real service.

The manifest handling has one more feature worth mentioning, because it is the kind of thing that only appears in a tool that has been used on real manifests. The tool supports elements that link one resource to another, but only with the on-load semantics, and it supports the variant that resolves to zero. In practice this matters for services that use indirection to move content between delivery networks, and the readme's specific limitation tells you the author knows exactly which part of the specification is implemented.

Cookies and subtitles

Two features stand out as being about the messy reality of streaming services rather than about the specification.

The first is cookies. The tool can read cookies from Firefox, Chromium, Chrome, Chrome Beta, Safari and Edge, on Linux, Windows and MacOS, and it does so through a separate library written for scraping browser profiles. The command line argument is a browser name, and the readme notes that browsers which support multiple profiles have all their profiles scraped for cookies.

Scraping every profile is a deliberate choice, and it is the right one. A service that requires a login will have set its cookie in whichever profile the person actually uses, and a tool that only read the default profile would fail for half the people who have more than one.

The second is subtitles, and the support is broader than a format list. The readme names the formats: WebVTT, TTML, SRT, tx3g, SMIL, and one more format for timed text within media fragments. And then it distinguishes two ways of publishing them.

A subtitle can be published as a complete file, which the readme calls a sidecar, downloaded once in one request. Or it can be segmented, made available in media fragments that are downloaded on demand, and the readme gives the codec identifiers for the two variants of that, one for timed text carried in fragments and one for the in-band variant.

The distinction is not academic. A service that publishes sidecar subtitles gives you one file, and any tool that can fetch a URL can get it. A service that publishes segmented subtitles requires the client to know the segment structure of the subtitle track, follow the same fragment pattern as the audio and video, and stitch the pieces back together in the right order with the right timing. That is a second timeline to track alongside the media, and the fact that the tool handles both means it works against either kind of service.

So the two features together cover the two most common reasons a naive downloader of streaming video fails in practice: the service wants your session cookie, and the service wants the subtitle track assembled the hard way.

Rewriting the manifest, decryption, and the stack

The last two features are the most unusual, and the stack underneath explains why they are possible.

The first is manifest rewriting. Before downloading, the tool can transform the manifest itself: replace the URL of certain segments with other URLs, or drop certain periods based on criteria such as the content delivery network or the server they come from, or the content language. The rewriting instructions are given as a user-supplied XSLT stylesheet, which is applied to the XML of the manifest, and there is also a convenience form for dropping elements based on an XPath specification. The feature depends on an external stylesheet processor being installed.

That is a genuinely capable design. Rewriting the manifest is how you select content that a service has split across several delivery networks, and doing it in the tool rather than by hand means you can script it. The dependence on an external processor is a cost, and it is the usual cost of adopting a standard language for the transformation instead of inventing one.

The second is decryption of protected streams, and the readme is specific about how it works. It requires either of two external applications to be installed, one from a well known suite of tools and one from Google. A decryption key is given on the command line, and the argument can be used several times when different streams use different keys, and a second argument selects which of the two applications to use. The readme explains the difference in capability between them: one is able to decrypt more types of media streams, including in particular WebM containers and more encryption formats, while the other mostly works with the common encryption scheme used in broadcast.

A key is supplied by the user, and the tool does not obtain one; that is worth stating plainly, because it is the difference between a tool that works with content you are entitled to decrypt and one that circumvents protection. The key handling is a small piece of the design and it is the piece with the most legal consequence for the person using it.

The dependency list tells you what kind of program this is. The asynchronous runtime with the multi-threaded scheduler, the tracing library with a subscriber that supports environment filters, JSON and formatted output, and the manifest library with features for fetching, compression, a signalling format, and the second and third versions of the HTTP protocol. The HTTP client is configured with the pure Rust TLS stack rather than the system one, with support for compression, SOCKS proxies, JSON, cookies, and the newer protocol versions, and the progress bar library, the argument parser with wrapped help text, and a small set of libraries for error handling, temporary files, file system errors with context, and units with prefixes.

And the repository layout is small and disciplined: a source directory, a tests directory, a documentation directory, a directory of ancillary files, a change log, a task runner file, and a manifest with the dependency and test sections separated in the conventional way. The container image size badge in the readme is there because the image is one of the distribution channels, and a small image is part of the value for a tool you might run once.

Editorial conclusion

dash-mpd-cli fits somebody who needs the file rather than the stream, whether that is an archive, a personal copy of something they are entitled to keep, or a fixture for a test, and it fits somebody whose service uses DASH with the less common manifest features. It does not fit a service that only publishes HLS, because the tool says plainly that it has no support for that. Verify first that the service you are pulling from uses a DASH manifest, that the container and codec you want are supported by the muxing tool you have installed, and that you are using a decryption key you are entitled to use if you are downloading protected content.

Frequently asked questions

Official sources

1. emarsden/dash-mpd-cli on GitHub
2. License: MIT
3. Project website
4. README
5. Releases