waymore pulls archived URLs from seven sources and keeps the responses too

Seven sources where waybackurls uses one

waymore is a URL collection tool for people doing reconnaissance on a domain, and it starts from an explicit comparison with two predecessors. The README credits waybackurls by TomNomNom for pulling URLs from web.archive.org plus an index collection on index.commoncrawl.org, and gau by lc for adding AlienVault, URLScan, VirusTotal and Intelligence X. waymore's claim is that it queries all seven of those sources itself.

The full list: Wayback Machine, Common Crawl, AlienVault OTX, URLScan, VirusTotal, GhostArchive, and Intelligence X, which the README marks as academia or paid tiers only. That marker is a practical constraint rather than a footnote, since Intelligence X results depend on an account tier you may not have.

The second difference the README puts first is more interesting than the source count. waymore can download the archived responses for Wayback Machine and URLScan URLs, which means you end up with the archived page bodies on disk rather than a bare list of links. Once you have those, you can search them for further links, developer comments and extra query parameters, which is where most of the additional value sits in a real engagement.

The third difference is honesty about failure. The README says other tools do not deal with the rate limiting now in place at these sources and will often stop with incomplete results without telling you the results are incomplete. Silent truncation is the worst possible behaviour for recon, because downstream work assumes the list is complete.

Installing with pip, pipx or a container

Installation is a normal Python package install, and the README is explicit that it wants the default global environment rather than a virtual one:

pip install waymore

Installing from source works too, with verbose output for debugging:

pip install git+https://github.com/xnl-h4ck3r/waymore.git -v

and upgrades use `pip install --upgrade waymore`. For an isolated install the README points at pipx and gives the git form, which is the route to prefer if you would rather not touch your global site packages:

pipx install git+https://github.com/xnl-h4ck3r/waymore.git

The repository also ships a Dockerfile with a two stage build on python:3.12-slim, a `docker-entrypoint.sh`, a non-root `appuser`, and a `results` directory created and chowned at build time, so a container deployment is a supported path rather than something a user assembled.

One documentation conflict is worth knowing about, because it affects which Python you need. The README says waymore supports Python 3.7 or higher, explaining that 3.7 is the floor for async and await support. The packaging metadata in the repository says something different: `pyproject.toml` declares `requires-python = ">=3.9"`, the Ruff and Black configuration both target `py39`, and the Dockerfile builds on Python 3.12. Two project files agree with each other against one README sentence, so treat 3.9 as the real minimum and let the metadata decide.

Input as a domain, a path or a TLD, and why subdomains hurt

The `-i` or `--input` argument takes a domain, or a file of domains, or a domain with a path. Two behaviours are worth internalising because they change the shape of your results.

First, schemes, port numbers, query strings and fragments are stripped from what you pass in. Second, adding a path narrows the search to that path, which is useful for depth on one area and useless for coverage. If `-i` is a bare domain, `-mode` defaults to `B`, meaning both URLs and responses. If `-i` includes a path, `-mode` defaults to `R`, responses only.

You can also pass a TLD by prefixing with a period, so `.mil` returns subdomains across all domains with that extension. The README flags one exception in that mode: AlienVault OTX is excluded, because it needs a full domain rather than a TLD.

The most emphatic warning in the README is about input shape. Passing a file of subdomains to get everything for a domain is called out as a common mistake, with an instruction not to do it: pass the domain and you get all its subdomains, far more quickly, without missing anything. It is a good example of a tool author knowing their users' habits well enough to pre-empt the specific wrong thing they keep doing.

Output paths are explicit through `-oU` for the links file and `-oR` for the responses directory. If you pass neither, waymore creates a `results` directory under the path in the `DEFAULT_OUTPUT_DIR` key of `config.yml`, typically `~/.config/waymore/`, and nests a directory named after the target inside it.

Config files, defaults and the upgrade path

Behaviour is driven by `config.yml`, which the repository ships and `setup.py` copies into a platform specific location at install time. The copy step is platform aware: `%APPDATA%/waymore` on Windows, `~/Library/Application Support/waymore` on macOS, and `~/.config/waymore` elsewhere. That same path is where the README says to look for `DEFAULT_OUTPUT_DIR`.

Upgrades carry a specific warning. If a `config.yml` already exists it is not overwritten; a `config.yml.NEW` is written beside it instead. To adopt the new defaults you delete the old file and rename the `.NEW` one back. The rename dance is visible in `setup.py`, which rotates the existing file to `.OLD`, copies the new one in as `.NEW`, and then swaps them into position so your settings survive an upgrade. That is careful behaviour for a tool people run against live third party APIs.

A gap was closed in v8.7: if `config.yml` is missing from the expected directory it is now created with default values automatically. The README notes this specifically fixed installs done with pipx, which had not been creating the file at all. For anyone whose first run failed on a missing config, that release note is the answer.

The dependency list is short and worth a glance: PyYAML, requests, termcolor, psutil and tldextract. v8.8 removed aiohttp and urlparse3 from it after finding they were never imported, since URL parsing uses the standard library `urllib.parse` and concurrency had moved elsewhere.

Coverage over speed, and the timeouts that make it survivable

The README states the tradeoff in capitals: the tool can be very slow, and it is meant for coverage, not speed. Combined with the advice against subdomain input files, the intended usage pattern is clear: a handful of carefully chosen high value domains, run thoroughly, rather than a wide sweep of hundreds of hosts.

What that slowness comes from is visible in the v8.8 notes. Many outbound HTTP requests were previously made with no timeout at all, including requests for the number of pages to search, the URLScan, VirusTotal and Intelligence X API calls, the Common Crawl index collection lookup and its streaming download of index data, and the Discord and Telegram completion notifications. A slow source could block a worker thread indefinitely, and the symptom was waymore appearing to hang with no way to recover short of killing it.

v8.8 fixed that with a `TimeoutSession` wrapper around `requests.Session` applied to every request, defaulting to a `DEFAULT_TIMEOUT` of 30 seconds where nothing was already specified. Calls that set their own timeout, including archived response downloads controlled by `-t` or `--timeout`, keep their value. That is the right shape of fix, since response downloads are the slow part and need a longer budget than an API ping.

Notifications to Discord and Telegram on completion exist, which is a reasonable accommodation for a job you start and walk away from.

Version 8.9 fixes a filename bug from analytics beacons

The most recent release, v8.9 on 2026-06-11, is a single bug fix and it is a good one. When saving archived responses, the file extension is derived from the URL path. For certain URLs, specifically captured analytics or beacon requests with long query string like values after a dot in the path, that produced an extremely long extension which got appended to the filename, and writing the file failed with `File name too long`.

The fix validates the extension length and treats anything over 20 characters as invalid, setting it to `unknown`. The check was applied to both the Wayback archive and the URLScan response saving paths.

This is worth dwelling on for a second, because it shows what response downloading actually costs you. An archived response is a saved file whose name comes from an attacker-influenced URL, and a single beacon pixel with a long token in the path is enough to break a save. Any tool that writes to the filesystem based on remote URLs needs this class of guard, and getting a 20 character limit is exactly the kind of specific fix that only comes from someone hitting the error against real captures.

The repository is MIT licensed, has only a handful of open issues, is not archived, and was last pushed on 2026-06-11.

Editorial conclusion

waymore earns its place when a bug bounty or an investigation needs every archived URL for a domain rather than a sample, because the archived response download is the part you cannot easily reproduce by hand and the rate limit handling is the part you cannot reproduce at all. It is the wrong tool for a quick single subdomain check, since the README warns at length that it can be very slow because it targets coverage instead of speed, and it is the wrong tool when you lack API keys for the sources you care about most. Start with `pip install waymore`, run it in URL mode first to see what a domain yields before committing to a response download, and read the argument table rather than guessing flags, since the TLD mode quietly excludes AlienVault OTX.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. xnl-h4ck3r/waymore on GitHub