projectdiscovery/httpx: the Go HTTP probing toolkit, not the Python client

What projectdiscovery/httpx is for, and who ends up using it

The README describes httpx as a fast and multi-purpose HTTP toolkit that runs multiple probes using the retryablehttp library, and says it is designed to maintain result reliability with an increased number of threads. The unit of work is a host, a URL or a CIDR range, and the output is a set of facts about each one. That shape fits reconnaissance and asset inventory: you have a list, you want to know which entries answer, what they answer with, and what is behind them. The topics on the repository (bugbounty, pentest-tool, osint, cybersecurity) match that audience, but the tool is not limited to security work. Any pipeline that needs a per-host HTTP fingerprint, such as a certificate expiry sweep or a check for which internal hosts still respond, can use the same flags.

The distinction that matters most is the name. There is a widely used Python HTTP client also called httpx, and search results for the term mix the two. This project is a Go binary that you run from a shell; it is not a library you import into a Python program, and it does not give you an async client object. If you arrived looking for `httpx.AsyncClient`, you are on the wrong project.

How the probing pipeline actually works

Input arrives three ways: a file of hosts via `-l`, one or more targets via `-u`, or a raw request file via `-rr`. The README also lists `-im` for input mode, with burp as the documented value, which covers pasting a saved request rather than a bare host list. Hosts, URLs and CIDR ranges are all accepted, and CIDR input is expanded internally (the go.mod pulls in mapcidr for that).

Each input then goes through a set of probes. The README's table marks which run by default and which need a flag. URL, title, status code, content length, TLS certificate, CSP header, line count, location header, web server, websocket, response time, IP, CNAME, word count, body hash, header hash, request method and URL scheme are on by default. Raw HTTP, HTTP2, HTTP pipeline, virtual host, favicon hash, JARM hash, paths, ports, CDN, ASN, redirect chain, probe status and request method probes are off unless you ask for them. That default set is the reason a bare run is already useful, and also the reason a bare run is slower than you might expect on a large list.

Two behaviours are worth calling out. First, the README states there is a smart auto fallback from https to http by default, so a host that does not speak TLS on 443 still gets reported over plain HTTP rather than dropped. Second, the retryablehttp dependency is not cosmetic: the README says the tool handles edge cases by doing retries and backoffs for handling WAFs. That is a deliberate trade of latency for completeness, and it is why rate limiting flags exist alongside the thread count.

The result is a stream of records, one per probed host, that you can send to JSON or CSV and hand to the next stage.

Installing projectdiscovery/httpx and running a first probe

The README states that httpx requires go >=1.25.0 to install successfully, and gives one command. Note that go.mod declares `go 1.26.0`, so the module itself expects a newer toolchain than the README's stated floor; check what your environment provides before starting.

go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest

The README points to https://docs.projectdiscovery.io/tools/httpx/install for other installation methods, so treat the command above as the source-build path rather than the only one. A Dockerfile also exists in the repository and builds the binary from `./cmd/httpx`, ending with an `ENTRYPOINT ["httpx"]`.

Once the binary is on your PATH, confirm the flags your build supports:

httpx -h

The README notes this displays help for the tool and lists all supported switches. If the output looks nothing like the flag list in the README, you have installed a different program with the same name.

For a first real use, take a small file of hosts and ask for status code, title and the technology fingerprint. The README's usage section shows the invocation form and the flags it accepts:

httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.

Usage:
  ./httpx [flags]

Flags:
INPUT:
   -l, -list string              input file containing list of hosts to process
   -rr, -request string          file containing raw request
   -u, -target string[]          input target host(s) to probe
   -im, -input-mode string       mode of input file (burp)

The `-l`, `-u` and `-rr` flags are the input paths described above, and `-im` selects the input file mode. The probe flags are listed separately in the same help output, under a PROBES heading, so check there for the exact spelling of the probe you want before building a pipeline. Hosts that do not respond produce no record rather than an error line, so an empty output file means nothing in your list answered, not that the run failed.

For a larger list, the README's flags give you two separate controls: thread count for concurrency and rate limiting for how hard you hit a target. Use both. A run that only raises threads will complete faster and also trip more WAFs, which is exactly the case the retry and backoff logic was written for.

Where projectdiscovery/httpx is the wrong tool

The README carries an explicit disclaimer: the project was primarily built to be used as a standalone CLI tool, and running it as a service may pose security risks, with the recommendation to use it with caution and additional security measures. If your plan is to wrap it in a long-running daemon that accepts arbitrary targets over HTTP, you are outside the intended design and you own the consequences. There is no authentication layer described for such a deployment, and the tool will happily make requests to whatever it is pointed at.

The second limitation is the one that trips people up most often: this is not an HTTP client library for Go or Python. If you need to make a request from inside your application, handle cookies, follow redirects with custom policy, or manage a connection pool, httpx gives you none of that as an importable API. It is a binary that reads input and writes output. Trying to shell out to it per request is slower and more fragile than using a client library directly.

The third is scope. The probes are HTTP-level facts. There is no crawl, no form submission, no JavaScript execution except the headless screenshot path, and no vulnerability detection. The `-ss` screenshot flag does drive a headless browser (the go.mod includes go-rod, and the Dockerfile installs chromium), but that is a rendering path for images, not a general browser automation tool. If your question is "what does this page do when clicked", httpx is not the instrument.

Finally, the README warns that the project is in active development and that you should expect breaking changes with releases, with a note to review the changelog before updating. Pin a version in CI rather than tracking latest.

How it differs from the Python httpx client and from curl-based scripts

The most common alternative people land on is the Python httpx library, and the difference is categorical rather than incremental. The Python library is an HTTP client you call from code: you construct a client, issue requests, and handle responses in your own process. projectdiscovery/httpx is a program you feed a list to; it owns the concurrency, the retries and the output format, and you consume records afterwards. If your task is one request inside an application, the Python library is the right shape. If your task is ten thousand hosts and a table of results, the CLI is the right shape, and writing that loop yourself in Python means reimplementing the retry, backoff and rate-limit behaviour the README describes.

The second alternative is a shell script around curl. That works for a handful of hosts and falls over on the details: TLS certificate fields, CNAME and ASN lookups, JARM and favicon hashes, and technology detection against the wappalyzer dataset are all things the README lists as probes. Reproducing the JARM hash in a shell loop is not a weekend task. Where curl wins is small, precise, scripted requests with exact header control, and the README does expose a raw HTTP probe plus a `-rr` raw request file for the cases where you need that precision inside httpx itself.

Maintenance, releases and what the MIT licence means here

The repository is not archived, and the last push was on 2026-09-16. Releases are frequent and versioned: v1.12.0 on 2026-09-08, v1.11.0 on 2026-08-31, and v1.10.0 on 2026-07-09. That cadence is the upgrade cost. The README explicitly warns to expect breaking changes with releases and to review the changelog before updating, so an unpinned install in a CI job is a future outage. Pin a tag, read the changelog for the jump you are making, and rebuild.

The build itself is ordinary Go. The Makefile defines `build`, `test` and `tidy` targets, with `build` producing a binary named httpx from `cmd/httpx/httpx.go`. The Dockerfile builds on `golang:1.26.5-alpine` and runs on `alpine:3.18.2`, installing bind-tools, ca-certificates and chromium in the runtime image. That chromium install is what makes the image large; if you never use `-ss`, the screenshot capability is dead weight in your container.

On licensing: the project is MIT, and the README links to the MIT licence text. MIT is permissive, which generally means you can use, modify and redistribute the code provided the copyright notice and licence text are preserved, but this is a description of the licence, not legal advice. Note that the binary links a long list of third-party Go modules, each with its own licence, so a redistribution review should cover the dependency tree and not just the top-level LICENSE.md. If your organisation has a policy on shipping binaries with bundled dependencies, that review is the gate, not the MIT label on the repository itself.

Editorial conclusion

Adopt projectdiscovery/httpx when you already have a list of hosts, URLs or CIDR ranges and need structured per-host facts for a pipeline. Do not adopt it as an HTTP client inside a Go program, and do not expose it as a network service: the README states it was primarily built as a standalone CLI and that running it as a service may pose security risks. Before rolling it out, verify the Go toolchain version your build environment provides against the go directive in go.mod, and check the changelog for the release you pin, because the README warns to expect breaking changes between releases.

Frequently asked questions

Official sources

1. License: MIT
2. projectdiscovery/httpx on GitHub
3. Project website
4. README
5. Releases