Down streams a remote file into a Tempfile and stops early on a size cap

Two entry points, one of which never holds the whole file

Down is a utility tool for streaming, flexible and safe downloading of remote files, and it exposes two primary methods.

Down.download fetches the whole file into a Tempfile.

require "down"

tempfile = Down.download("http://example.com/nature.jpg")
tempfile #=> #<Tempfile:/var/folders/k7/6zx6dx6x7ys3rv3srh0nyfj00000gn/T/20150925-55456-z7vxqz.jpg>

Down.open returns a Down::ChunkedIO that represents the remote file and retrieves content as it is being downloaded. When you read from it, Down internally downloads chunks of the remote file, but only how much is needed.

remote_file = Down.open("http://example.com/image.jpg")
remote_file.size # read from the "Content-Length" header

remote_file.read(1024) # downloads and returns first 1 KB
remote_file.read(1024) # downloads and returns next 1 KB

remote_file.eof? #=> false
remote_file.read # downloads and returns the rest of the file content
remote_file.eof? #=> true

remote_file.close # closes the HTTP connection and deletes the internal Tempfile

The difference matters in a specific situation. With download, the whole body passes through your process before you get anything back. With open, a request handler can start forwarding bytes to a client while the transfer is still in progress, which is the difference between proxying a large file and buffering it.

The IO methods that ChunkedIO implements are listed explicitly, and the list is what tells you it is usable as a drop-in IO.

* `#read` & `#readpartial`
* `#gets`
* `#seek`
* `#pos` & `#tell`
* `#eof?`
* `#rewind`
* `#close`

Having seek, pos and rewind is what makes it a real IO rather than a stream, and that comes with the caching model discussed below.

The size cap aborts on the header, not after the body

The maximum size feature is the one to read closely, because the reason it exists is stated plainly. When you're accepting URLs from an outside source, it's a good idea to limit the filesize, because attackers want to give a lot of work to your servers.

Down.download("http://example.com/image.jpg", max_size: 5 * 1024 * 1024) # 5 MB
# Down::TooLarge: file is too large (max is 5MB)

And then the README asks the obvious question, what is the advantage over simply checking size after downloading, and answers it. Down terminates the download very early, as soon as it gets the Content-Length header. And if the Content-Length header is missing, Down terminates the download as soon as the downloaded content surpasses the maximum size.

That second sentence is the part that makes the feature actually safe rather than decorative. A server can lie about Content-Length, or omit it, and in the first case the header check stops the transfer before any body is read, and in the second case the running total stops it. A check after the download would have already paid for the bandwidth and the disk write.

The exception hierarchy makes the error surface explicit. Down tries to recognize various types of exceptions and re-raise them as one of the Down::Error subclasses, and the hierarchy is listed in full.

* `Down::Error`
  * `Down::TooLarge`
  * `Down::InvalidUrl`
  * `Down::TooManyRedirects`
  * `Down::NotModified`
  * `Down::ResponseError`
    * `Down::ClientError`
      * `Down::NotFound`
    * `Down::ServerError`
  * `Down::ConnectionError`

So a caller can rescue Down::NotFound to mean the resource is absent, Down::TooLarge to mean the input was too big, and Down::ResponseError to mean anything else the server rejected. That is a normal, well shaped library error surface, and it is what you want when the caller is a controller that has to turn a remote failure into a sensible HTTP status.

Three HTTP backends, and the choice is yours

The README says Down can use open-uri plus Net::HTTP, http.rb or HTTPX as the backend HTTP library, and that choice is the first thing to make.

The open-uri plus Net::HTTP option is the default and it is the one that needs nothing beyond the standard library, which matters for a gem that is easy to add to an existing application without pulling a new HTTP stack into the dependency tree. http.rb is a dedicated HTTP client gem, and HTTPX is another. The choice of backend is a real decision, because the three differ in how they handle redirects, timeouts, and streaming semantics, and the README does not spell out the differences.

The Ruby side of the ecosystem makes that choice less unusual than it would be elsewhere. Ruby has had several generations of HTTP client, and a library that abstracts over them lets an application pick the one that fits its logging, instrumentation and timeout conventions.

Other things the gem handles that you would otherwise write yourself. Basic authentication from the URL is detected and applied automatically, so Down.download and Down.open work with a URL of the form http://user:[email protected] without any extra configuration.

The response metadata is attached to the resulting Tempfile.

tempfile.content_type      #=> "text/plain"
tempfile.original_filename #=> "document.txt"
tempfile.charset           #=> "utf-8"

That is a small convenience with a real payoff, because the content type and the original filename are the two values a download handler almost always needs and they come from the response rather than from a second request.

And the response status and headers are reachable from the open object.

remote_file = Down.open("http://example.com/image.jpg")
remote_file.data[:status]   #=> 200
remote_file.data[:headers]  #=> { "Content-Type" => "image/jpeg", ... } (header names are normalized)
remote_file.data[:response] # returns the response object

The README notes that a Down::ResponseError exception is automatically raised if the response status was 4xx or 5xx, so an error response never reads as an empty file.

Caching is the reason the IO interface is complete

The caching behaviour is the design decision that explains the whole of the ChunkedIO class.

By default the downloaded content is internally cached into a Tempfile, so that when you rewind the ChunkedIO, it continues reading the cached content that it had already retrieved.

remote_file = Down.open("http://example.com/image.jpg")
remote_file.read(1*1024*1024) # downloads, caches, and returns first 1MB
remote_file.rewind
remote_file.read(1*1024*1024) # reads the cached content
remote_file.read(1*1024*1024) # downloads the next 1MB

That is what makes seek, pos and rewind work at all. Without a cache, a rewind would have to reissue the request, and a seek to an arbitrary offset would have to either restart the transfer or issue a range request, neither of which the README describes.

The cost of the cache is disk and the possibility of a large temporary file for a file that is only partially read. The README gives the escape hatch and the condition for using it. If you want to save on IO calls and on disk usage, and don't need to be able to rewind, you can disable caching with rewindable: false.

Down.open("http://example.com/image.jpg", rewindable: false)

There is a third mode, which is more efficient than either. You can yield chunks directly as they're downloaded via #each_chunk, in which case the downloaded content is not cached into a file regardless of the rewindable option.

remote_file = Down.open("http://example.com/image.jpg")
remote_file.each_chunk { |chunk| ... }
remote_file.close

So the three levels are, download for a whole file with the metadata attached, open with caching for a seekable IO, and open with each_chunk for a forward-only stream with no disk use.

ChunkedIO is generic, so it is not only about HTTP

The part of the README that surprises people is that Down::ChunkedIO is not tied to HTTP at all.

The Down.open method performs the HTTP logic and returns an instance of Down::ChunkedIO, but ChunkedIO is a generic class that can wrap any kind of streaming. It accepts an Enumerator that yields chunks of content, and provides an IO-like interface over that enumerator, calling it whenever more content is needed.

require "down/chunked_io"

Down::ChunkedIO.new(...)

The constructor options are listed and they are the whole contract.

* `:chunks` – `Enumerator` that yields chunks of content
* `:size` – size of the file if it's known (returned by `#size`)
* `:on_close` – called when streaming finishes or IO is closed
* `:data` - custom data that you want to store (returned by `#data`)
* `:rewindable` - whether to cache retrieved data into a file (defaults to `true`)
* `:encoding` - force content to be returned in specified encoding (defaults to `Encoding::BINARY`)

The `:data` option is the one that makes the class genuinely reusable, because it is how you attach your own metadata to the stream and get it back. The README gives the example, a streaming IO over a MongoDB GridFS file.

require "down/chunked_io"

mongo = Mongo::Client.new(...)
bucket = mongo.database.fs

content_length = bucket.find(_id: id).first[:length]
stream = bucket.open_download_stream(id)

io = Down::ChunkedIO.new(
  size: content_length,
  chunks: stream.enum_for(:each),
  on_close: -> { stream.close },
)

So if your application already has a storage system with a chunked read interface, you can hand it to ChunkedIO and get the same IO semantics, the same caching behaviour and the same close handling that the HTTP path gives you. The gem is a download library, but the class underneath it is a general streaming adapter.

Where Down is the wrong choice

The first wrong case is a resumable or parallel download. The README describes streaming, a size cap, a destination, progress callbacks and a cache. It does not describe range requests, retry with backoff, or splitting a file into parts. If you are fetching a very large file over an unreliable link and want to resume from where you left off, that is a different class of tool.

The second wrong case is a download that should never touch local disk. The caching model writes to a Tempfile by default, and even the forward-only mode still needs the tempfile unless you use each_chunk, and the Tempfile is deleted on close. For a service that streams straight to object storage, the Tempfile is a copy of the file on the local disk, which on a small instance is a real cost for large objects.

The third wrong case is a project that wants one HTTP client everywhere. The gem supports three backends and makes the choice yours, which is the right call for a general purpose gem and the wrong one for an application that has standardised on a single client with its own instrumentation, timeout policy and connection pooling. In that case, using the client directly and writing the streaming logic yourself is often the shorter path.

The fourth wrong case is expecting an actively maintained gem with published releases. The repository has no GitHub releases, the last push to master was on 2026-05-06, and the CHANGELOG.md at the root is the version history. The installation instruction pins the major line, gem "down", "~> 5.0", so a team that pins a specific patch version is on its own with respect to upstream fixes.

How it differs from open-uri and Faraday

The comparison worth making is with what a plain Ruby application would otherwise do, and the difference is a set of small behaviours that each save a bug.

open-uri gives you a file-like object for a URL. What it does not give you is a size cap that aborts on the Content-Length header, a response metadata object, a progress callback, or an exception hierarchy that distinguishes a 404 from a 500 from a connection failure. The typical open-uri based handler checks the size after the fact, which is the exact thing the README identifies as the wrong order.

Faraday and similar middleware-based clients are the other alternative. They are the right tool when you want a request/response pair with a specific adapter. What they do not give you is the IO interface over a partially downloaded body, with seek, rewind and a cache, and the size limit enforced during the transfer.

The difference in approach is that Down is a download utility rather than an HTTP client. It is concerned with one operation, getting a remote file into your process safely, and it treats the HTTP details as an implementation detail behind a pluggable backend. The backend choice means you can use the HTTP client that your application already has, and the generic ChunkedIO means the same semantics apply to non-HTTP sources.

The third difference is the error surface. Down normalises everything into Down::Error subclasses, which means a caller can write a rescue that distinguishes a missing file from an oversized one from a server failure without inspecting HTTP status codes at four different call sites.

Editorial conclusion

Down fits a Rails application that accepts a URL from a user and has to fetch the file without letting the request hold a socket open for the whole transfer, and it fits a service that needs to enforce a size cap before the body is received rather than after. It does not fit a project that needs parallel or resumable downloads, since the README describes no such support, and it does not fit a project that wants the file to end up in object storage without a local temp file, since the caching model writes to a Tempfile by design. Before adopting it, decide which backend you want, because the backends differ in behaviour and the gem supports open-uri with Net::HTTP, http.rb and HTTPX, and note that the repository has no GitHub releases and the last push was on 2026-05-06.

Frequently asked questions

Official sources

1. Issues
2. janko/down on GitHub
3. License: MIT
4. README