Cloudflare-Datamining

What gets tracked

The project mines a wide range of public Cloudflare surfaces. It extracts API schemas from api.cloudflare.com and watches them change, follows blog post content, tracks cdn cgi endpoints and request.cf keys, and records dashboard subroutes and translations that hint at undocumented APIs and new features. It also watches the DOH schema returned by 1.1.1.1, domains Cloudflare owns, active A/B experiment gates, Cloudflare GitHub repos, job listings, marketing site JSON, NPM packages and workerd capnp schemas.

Structure and output

Scripts live in a scripts directory and the extracted data lives in a data directory, with automated commits made frequently so the history itself documents change. A subset of the data is visualized on the cfdata.lol website, and a Discord server sends notifications when things change. The registrar tracker follows Cloudflare Registrar available TLDs and pricing over time, and the entitlements list records known account and zone entitlements, though that part is not automated.

Contributing data

Pull requests are welcome for improvements to the scripts and data. The maintainer invites people who run an interesting Cloudflare account to contribute their account or zone entitlements as a JSON dump, shared through Discord or Twitter, in order to grow the known list. The work is presented as a public, frequently updated record of how one of the largest edge networks presents itself across its many surfaces.

Editorial conclusion

Cloudflare datamining is written in HTML and has gathered about 198 stars. Its change feeds publish through automated commits and a Discord notification channel.