wget2 rebuilds GNU Wget around libwget, and splits the license in two

A rewrite, not a patch set on the old downloader

GNU Wget2 is described in its own README as the successor of GNU Wget, a file and recursive website downloader, and the text is emphatic that it was designed and written from scratch. That is the first thing to understand, because it explains the option differences people hit when they swap one for the other. This is not wget1 with new flags bolted on; it is a different program with a different internal shape that keeps the same job.

The structural change is the split into two pieces. Wget2 wraps around libwget, and the README says libwget provides the basic functions needed by a web client. The program you run is the client built on top of that library. So the project has two deliverables with two audiences: a downloader for people who want a command line tool, and a library for people who want the same HTTP, TLS and cookie behaviour inside their own code.

The README also states plainly that Wget2 works multi-threaded and uses many features to allow fast operation. The performance argument it makes is specific: in many cases Wget2 downloads much faster than Wget1.x due to HTTP2, HTTP compression, parallel connections and use of If-Modified-Since HTTP header. That is a claim from the project itself, not an independent measurement.

If your workload is a handful of files, the rewrite buys you little. The rewrite pays off on mirrors, multi-hundred-megabyte downloads, and sites that respond well to conditional requests.

libwget is the part other projects are meant to embed

The examples directory is the clearest signal of intent. It contains batch_loader.c, http_get.c, http_get2.c, http_multi_get.c, check_url_types.c, print_css_urls.c, print_html_urls.c, relative_to_absolute_url.c, websequencediagram.c, and getstream.c, which the README ties to ICEcast and SHOUTcast support via library.

These are not sample scripts for downloaders. They are demonstrations of a client library being driven from C. http_multi_get.c in particular suggests the concurrency model of the library is exposed to callers rather than buried in the tool, and getstream.c shows streaming media handled as a library feature.

That structure also explains why libwget.pc.in exists at the top level of the repository. It is a pkg-config template, which is how a library advertises its headers and link flags to a build system, so the project is set up to be consumed by other packages rather than only invoked from a shell.

The practical consequence for an adopter is a licensing decision. The README states GNU Wget2 is licensed under GPLv3+, while libwget is licensed under LGPLv3+. If you link libwget into a proprietary program, the LGPL terms and the COPYING.LESSER file at the repository root govern that use, and GPL terms govern the tool itself. That split is a design choice, and it is the reason the two components carry different license identifiers.

Compression codecs beyond gzip and deflate

Most downloaders handle gzip and deflate and stop there. Wget2 lists a wider set: brotli decompression support with the Accept-Encoding: br header, zstandard support under RFC8478 with Accept-Encoding: zstd, lzip support, bzip2 Content-Encoding and Accept-Encoding compression type, and XZ Content-Encoding support.

For a mirrored site this matters more than it looks. A server that advertises brotli or zstd will send compressed bodies, and a client that does not understand those codings either fails or receives bytes it cannot use. Supporting them means fewer stalls on content-heavy sites and smaller transfers.

Every one of those codecs is optional at build time, with a matching optional library in the requirements list: liblzma for lzma, libbz2 for bzip2, libbrotlidec and libbrotli, and libzstd. Missing the libraries does not break the build, it removes the feature.

Character handling is a separate axis. The README notes that the character encoding of input files may be specified despite from local and remote encoding through the --input-encoding option, and libiconv is listed as needed for IRI and IDN support. For archives of mixed-encoding pages, this is the option that keeps filenames and text consistent across a mirror.

A TLS layer built on GnuTLS with pinning, HSTS and resumption

The TLS story is unusually detailed for a downloader. The README lists TLS Session Resumption including a persistent session data cache, TLS False Start with GnuTLS 3.5.0 or later, TCP Fast Open for plain text and for HTTPS, OCSP stapling plus OCSP server querying as a fallback marked experimental and needing GnuTLS 3.3.11 or later, and HPKP, HTTP Public Key Pinning under RFC7469, with a persistent database.

On top of that sit RFC 6797 HSTS support and the option --secure-protocol=PFS, which forces TLS only and additionally requires Perfect Forward Secrecy. That single option is the one to know about if you mirror a site over HTTP/2, since HSTS-only behaviour can be stricter than an arbitrary target needs.

Two dependencies shape what you can enable. libgnutls is required, at version 3.3, 3.5 or 3.6, and libwolfssl is optional if you would rather build against WolfSSL instead. libhsts is optional and enables support for HSTS preload lists, and libgpgme is optional and enables automatic signature verification.

The OCSP path is the one to read carefully: it is labelled experimental in the README, which is a fair signal to check it against your own environment before relying on it for certificate revocation behaviour in unattended mirrors.

Recursive mirroring that reads robots.txt, sitemaps and feeds

Recursive download of websites with or without spanning hosts is listed, and around that core sit the features that decide what a mirror actually captures.

Crawler politeness and scope come from respecting the robots.txt Robot Exclusion Standard and the META name="robots" tags, plus the support for scanning URLs from a local Sitemap XML file and for scanning sitemap files given in robots.txt, including gzipped sitemap XML, plain text sitemap indexes and sitemap index files.

Feed scanning is a separate path, with RSS 2.0 and Atom 1.0 support, each able to be forced on a local file rather than fetched over the network. So a document set can be assembled from a saved feed file without letting the tool follow every link on the page.

Metalink support covers RFC 6249 over HTTP for mirrors and hashes, the RFC 5854 download description format and .meta4 files, and Metalink 3, with checksumming done through libgnutls. That is the path for downloading from a mirror set with hash verification.

Cookies need care. The README describes session and non-session cookies, and detection of supercookies via the Mozilla Public Suffix List, using the new option --cookie-suffixes with a filename, and suggests putting it into ~/.wgetrc instead. Cookie policy is configurable rather than fixed, which means a default configuration can behave differently from what an operator expects on a shared host.

Building it means assembling a dependency list first

Wget2 is built from source with autotools, and the requirements list is long enough to plan around. The mandatory items are autotools (autoconf, autogen, automake, autopoint, libtool), python, which the README recommends for a faster bootstrap, rsync, tar, makeinfo from texinfo, pkg-config 0.28 or later, doxygen for documentation, pandoc for the wget2 man page, gettext 0.18.2 or later, libiconv, libz 1.2.3 or later, libgnutls, libidn2 0.14 or later, flex 2.5.35 or later, libpsl 0.5.0 or later, and libiconv for IRI and IDN support.

The core codec set follows the same shape:

liblzma >= 5.1.1alpha
libbz2 >= 1.0.6
libbrotlidec/libbrotli >= 1.0.0
libzstd >= 1.3.0
libgnutls (3.3, 3.5 or 3.6)
libidn2 >= 0.14
libpsl >= 0.5.0
libnghttp2 >= 1.3.0

Everything else in the list is optional, and each optional library buys a specific capability rather than a general improvement. libnghttp2 1.3.0 or later is optional and gates HTTP/2 support, libmicrohttpd 0.9.51 or later is optional and needed to run the test suite, lcov is for coverage reports, libpcre or libpcre2 is for regex filtering, and lzip is only needed to build distribution tarballs.

The README states the listed versions are recommended but older versions may also work, and it points to an online reference for command details. The repository root carries bootstrap, bootstrap.conf, configure.ac, Makefile.am and cfg.mk, so the source tree carries its own build machinery, and the build instructions themselves live in the README sections covering downloading and building from tarball and building from git.

Options that moved: the migration surface for wget1 scripts

Any script written for wget1 needs checking, because a from-scratch rewrite does not guarantee the same option names or the same defaults. The documented options that are most likely to matter in an existing script are these:

--chunk-size
--secure-protocol=PFS
-k/--convert-links
-K/--backup-converted
--input-encoding

The --chunk-size option drives multithreaded download of single files, and the number of parallel download threads is adjustable separately, so a large single asset and a large directory are two different tuning problems. Link conversion and backup of converted files are both present, which matters when a mirror has to preserve a working local copy of a rewritten page.

Configuration is file driven rather than flag driven. The README mentions an include directive for config files with wildcards allowed, so a shared ~/.wgetrc can pull in per-site fragments. That is the pattern to use if you maintain mirrors for several hosts, since it keeps the per-host differences out of the command line.

The README does not publish a one-to-one mapping from wget1 options to wget2 options, and it does not claim full backward compatibility. Treat any existing wget1 wrapper script as something to re-verify line by line, especially around retry and continuation behaviour, which the README does not enumerate.

Development happens on GitLab, not on the GitHub mirror

The GitHub repository you are looking at is not the primary home of the project. Its description says contributions are preferred at https://gitlab.com/gnuwget/wget2, but accepted here as well. The README links the bug tracker, development tree, online reference, coverage and fuzz coverage reports all under the gitlab.com/gnuwget/wget2 and gnuwget.gitlab.io addresses, and the build status badge points at GitLab pipelines.

So the practical rule is that code and bug reports go to GitLab, while the GitHub copy serves as a mirror that is also used for contribution intake, including Hacktoberfest-style intake given the hacktoberfest topic tag.

Development activity is visible in the release record. Version v2.3.0 was released on 2026-09-21, v2.2.1 on 2025-12-30, and v2.1.0 on 2023-08-31, and the last push to this repository was on 2026-10-18. That gap between v2.1.0 and v2.2.1 is worth noting, since it shows the release cadence has not been uniform, while the recent push date shows the tree itself is being worked on.

The test and verification apparatus is also part of the record: the README links a fuzzing status badge and separate coverage pages for unit and fuzz coverage, and the repository root holds fuzz/, tests/, unit-tests/, benchmarks/ and a ci-test.d directory, which tells you the project is exercised by continuous integration rather than by a manual checklist.

Editorial conclusion

wget2 suits anyone mirroring a site over HTTP/2 who wants parallel connections and a resumable client library they can link into their own program, and it is a better fit than wget1 for sites served with brotli or zstd. It is the wrong choice for a system that depends on classic wget1 option behaviour, since the option set is not a superset, and it is the wrong choice for anyone who wants a single self-contained binary without a long list of build dependencies. Verify first that your distribution ships a wget2 package, that the TLS backend you need (GnuTLS or WolfSSL) is present, and that the option names your existing scripts rely on still exist in the online reference before you migrate.

Frequently asked questions

Official sources

1. Issues
2. License: LGPL-3.0
3. README
4. Releases
5. rockdaboot/wget2 on GitHub