porn-domains: a blocklist that expects you to read its policy first

What the project is, stated carefully

The readme's own summary is the most careful paragraph in the repository, and it is worth quoting its structure even where the words are not repeated. The project is a community-maintained domain list for DNS-based filtering of adult content. It combines external sources with reviewed custom entries. It is not exhaustive, and it may contain errors.

That last sentence is unusual in a blocklist readme, and it is the right thing to say. A blocklist is a claim about the world, and the world changes; a list that does not say it is incomplete is telling the user something false.

Then there is the explanation of what DNS filtering can and cannot do, and it is the part that decides whether a deployment is a good idea. The filtering works at the domain level, not at the page or address level. Blocking a shared domain can affect unrelated content hosted on that domain. And the list cannot prevent access through other resolvers or through services that resolve names themselves.

The three sentences together describe the actual capability of the technique, which is coarse. You are not filtering pages; you are refusing to resolve names. Anything on a domain that also serves something legitimate goes down with it, and anything that resolves its own names internally is invisible to the mechanism.

So the readme's instruction to review the policy before using the list is not a formality. It is saying that a list of this kind makes decisions on your behalf, and that you should look at the criteria before you let it.

The project itself is a community effort with the usual apparatus: a logo, badges for contributors, sponsors, watchers, forks and stars, a licence, and a funding link. The topic tags place it squarely in the DNS filtering and family filtering world, alongside the hosts-file and resolver ecosystem it is designed to feed.

The file names change on purpose

The single most interesting engineering decision in the repository is that the generated files have hashed, random names, and that the readme tells you to resolve them through a metadata file rather than to hardcode an address.

The files are named with a content hash and then a random hash, and the readme states that the generated filenames are intentionally changeable, and that you should always read the metadata file to find the current ones, because a generated filename can change.

The reason is cache invalidation, and it is a good reason. A filtering list is fetched periodically by a resolver or by a script on a schedule. If the file name is stable, then every consumer that caches by address gets stale content, and the failure mode is silent: the resolver keeps serving the old list and nothing reports a problem. Changing the name on every regeneration turns a silent staleness into an immediate cache miss, which the consumer handles by downloading the new file. The random component matters too, because it defeats the caches that key on the content hash rather than on the path.

And the metadata file carries the information a consumer needs to make that work: the current filenames, the raw addresses to fetch them from, the update times, the line counts, and the file sizes. So the contract is: fetch the metadata, read the filename, fetch the file. Never hardcode the file name.

The readme links to the change log entry for the date the behaviour was introduced, which is the right way to point at a decision: here is when this changed, here is why, and here is what to do about it.

And the readme says the list is too large to view directly on the hosting site, so the repository's own page is not a place to read the list. It is a place to fetch artefacts from. Which is a good design, because a list this large is a machine input rather than a document, and the readme treats it that way.

The allowlist, which is the more interesting file

There are two generated lists, and the second one is the one a reader should look at first.

The first is the blocklist, generated from the sources and the custom entries. The second is an allowlist: the domains that are excluded from the blocklist.

That second file is the interesting one, because it is the counterweight to everything the readme warns about. A domain-level blocklist has exactly one failure mode that matters, which is a legitimate site sharing a domain with blocked content, and the allowlist is the mechanism for saying not this one. A list without an allowlist can only be conservative in one direction, and a conservative list is either too small to be useful or too large to be safe. The allowlist is what makes it possible to be both.

The three lists of resources in the readme are worth reading as a whole, because they show the project thinking about the whole life of a data artefact rather than only about its production. There is a policy page that says what qualifies for inclusion or removal, and it explicitly includes a section on machine-generated content, which is a real category for this kind of list and one that a maintained dataset has to have a rule about. There is a contributing page, for reporting a domain or proposing a change. There is a change log for the list structure and the automation. There is a frequently asked questions page, a support page, and pages for contributors and sponsors.

And the readme tells you to review the policy before using the list, which is what makes the policy a governing document rather than a reference page.

The dynamic badges on the file rows are worth a technical note too, because they are a small piece of engineering that most list projects do not do. Instead of a static badge, the badge fetches the metadata file and reports the current line count, the last update and the file size, and the address it fetches is the raw form of the metadata file with a query for the field it wants. So a reader can see at a glance, on the readme, how fresh the list is and how big it is, without opening anything. The raw file address points at the main branch, so the badge reflects the current state of the branch.

The disclaimer, and the licence

The end of the readme has a disclaimer with three bolded headings, and it is a good one, because each heading names a different kind of responsibility.

The first is no guarantee. The list may be incomplete, it may be out of date, and it may contain false positives, and you should review it before deploying it and test it in your own environment.

The second is that local laws and policies vary, that the project applies its own criteria, and that your organisation is responsible for deciding whether and how to use the list. That is the correct allocation of responsibility. The project maintains a list and publishes its criteria; the decision about whether a given list is appropriate for a given household or organisation belongs to the people accountable for it.

The third is use at your own risk, with the reason given: DNS blocking can disrupt access to legitimate content on a shared domain, and the maintainers are not responsible for decisions made using the list.

The licence is a share-alike creative commons one, and the readme gives the exact attribution text to use, as a markdown snippet.

[Bon-Appetit/porn-domains](https://github.com/Bon-Appetit/porn-domains) is licensed under [CC BY-SA 4.0](https://github.com/Bon-Appetit/porn-domains/blob/main/LICENSE).

Publishing the attribution string as a copy-and-paste snippet is a small piece of care that saves every user from getting the credit wrong, and a share-alike licence with a mandated attribution is one that is easy to comply with and easy to fail, so the snippet removes the failure mode.

The funding section is in the same spirit. The note for commercial users says that maintaining and reviewing a regularly updated dataset takes ongoing work, that commercial users are welcome to contribute improvements or to support the project financially, and it links to a donation page. Which is an honest framing of the economics: a list of this size is a dataset with a maintenance cost, and the way to pay that cost is either with time or with money.

The repository, and what is in it

The top level of the repository is small, and it is a good picture of how the project is organised.

There are the two generated list files at the root, with their hashed names, and a metadata file. Then there are two directories that hold the source of those files, one for the blocklist and one for the allowlist, so the generated artefacts at the root and the inputs that produced them are both in the tree.

And there is a documentation directory, which is where the policy, the contributing guide, the change log, the frequently asked questions page and the support page live. Which is the arrangement a project like this should have, and which a great many list projects do not have: the criteria that decide what goes in the list are versioned, reviewable and diffable alongside the data itself.

The remaining files are the standard ones, plus two that are worth a mention. There is a file of attributes for line endings and an editor configuration, which for a repository whose files are generated text consumed by other programs is not a trivial detail. And there is a directory for the continuous integration and release automation, which is what regenerates the two files, computes the hashes that end up in their names, and updates the metadata.

So the pipeline is: the source directories hold the reviewed entries, the automation combines them with the external sources, the generator writes the two files under hashed names, the metadata records what they are, and the readme tells consumers to read the metadata. Everything else in the repository exists to make that loop run reliably and to make the criteria behind it inspectable.

Which is the shape of a good data project. The artefact is large, machine-generated and constantly changing, and the value is not in the artefact but in the machinery and the policy around it. A reader who wants to know whether to trust the list should read the policy and the change log rather than the list itself, and the repository is arranged so that those two documents are the easiest things to find.

Editorial conclusion

The domain list fits a home network or an organisation that wants a coarse family filter in front of a resolver, and it fits somebody who wants to contribute reviewed entries. It does not fit a compliance requirement, because a domain list is the wrong instrument for one. Verify first how your resolver consumes the list, because the file names change and a hardcoded address will go stale, that you read the policy and the disclaimer, because the project is explicit that the list may be incomplete, may be out of date and may contain false positives, and that you accept the scope of domain-level blocking, since a single shared domain can take legitimate content down with it.

Frequently asked questions

Official sources

1. Bon-Appetit/porn-domains on GitHub
2. Issues
3. License: CC-BY-SA-4.0
4. README