diodb: a version-controlled dataset of vulnerability disclosure and bug bounty programs

A dataset with a directory in front of it

diodb is described in the README as the community database of every VDP and bug bounty program and its safe harbor status, now served live at directory.disclose.io. The primary language is listed as Python, which is a little misleading, because what the repository actually contains is data and a small set of tools rather than a Python package.

The README draws the distinction between the two things clearly. diodb is the open dataset, and directory.disclose.io is the live front end. The directory is the searchable, always-current place to look programs up, while the repository is the open, version-controlled dataset behind that story, and it remains open to contributions. The instruction to a reader is direct: if you just want to find a program, use the directory, and if you want to add or correct one in the open dataset, pull requests here are welcome.

The quick links table spells out the access paths. To search the database front end, disclose.io/programs. To download the raw database in .json format, the program-list.json file on the master branch of the repository. To generate your own vulnerability disclosure program, there is a separate policymaker.disclose.io tool. To learn more about VDPs, a separate repository called dioterms holds reference material, and there is a community forum at community.disclose.io.

The overall project frames diodb as part of the disclose.io project, described as the open, vendor-neutral infrastructure for vulnerability disclosure.

So the practical picture is a two-layer arrangement. A fast, searchable interface for looking things up, and a plain data file in git for anyone who wants to consume it programmatically or improve it.

Why the project says it exists

The About section lists four reasons, and they are worth quoting in substance because they define the intended audience.

diodb exists to drive the adoption of Safe Harbor for hackers and promote the cybersecurity posture of early adopters. It simplifies the process of finding the right contacts and channel at an organization. It helps both finders and vendors align around the expectations of engagement. And it provides a simple, vendor-agnostic point of engagement for program operators, potential program operators and the security community to maintain updates to their program.

The last one is the most interesting from a data design point of view. A program directory has two kinds of entries: programmes that want to be found, and the security community discovering which programme to approach. Maintaining both sides from one dataset means a programme operator and a researcher are looking at the same record.

The safe harbor angle is the other load-bearing idea. A disclosure programme that carries a safe harbor commitment tells a finder what protection they have, and having that status as a first-class field in the dataset is what makes the dataset different from a plain list of bug bounty pages. The topic tags on the repository match that framing, with vulnerability-disclosure, responsible-disclosure, safe-harbor-framework, bug-bounty, disclosure-policy, legal and security-research among them.

The homepage of the project is disclose.io/programs, and the licence on the repository is recorded as CC0-1.0.

Editing program-list.json and the one CI rule that catches people

The contribution flow is deliberately simple, and the README is explicit that one step causes most failed pull requests.

You edit program-list.json, then you format and sort the file, then you open a pull request. The formatting command is given as a shell snippet.

jq --indent 3 -s '.[] | unique_by(.program_name)' < program-list.json > _ && mv _ program-list.json

The README says CI enforces this and that it is the single most common reason a PR goes red. The command does two things. It re-indents the JSON to three spaces with jq, and it sorts the array and drops duplicates by program_name, which is what the unique_by call does before the output is written.

So the file has a canonical shape, and that shape is machine-checked. A contributor who edits one entry and skips the sort step will produce a diff that moves every record, which is both an unreadable review and a CI failure.

After the pull request, CI will validate the JSON, check for duplicates and verify the links. That last step is worth noting: a dead URL in a disclosure policy is a real problem for the people using the directory, so the link check is not decoration. The full guidelines live in CONTRIBUTING.md.

If you would rather not use git at all, the README offers alternatives: look a program up in the directory, or ask a question and discuss on the community forum. So the correction path does not require a GitHub account, though adding a missing program does.

What is actually in the repository

The top level of the repository is small and tells you the shape of the thing.

There is program-list.json, which is the dataset itself, and program-list.sqlite, which is the same content in a database form for anyone who wants to query it rather than parse it. There is program-list-schema.json, the schema that describes the records, and that file is the one to read first if you are writing a consumer, because it tells you the field names and their types without you having to infer them from examples.

Then there is a tools/ directory, a docs/ directory, a CODE_OF_CONDUCT.md, a CONTRIBUTING.md, a LICENSE file, a README.md and a .github directory for the CI configuration.

The combination of JSON, SQLite and a schema is a deliberate set of choices for a dataset meant to be consumed by other people. JSON is the editable source of truth and works fine in a diff, SQLite serves the kind of query where you want to filter across many records, and the schema file makes the shape explicit so a consumer is not parsing sample rows and hoping.

The project also maintains the disclose ecosystem as a set of separate repositories, with the dioterms repository for learning about vulnerability disclosure programs and the policymaker tool for generating your own, and a community forum for discussion.

Licence ambiguity, and the limits of a crowd-maintained directory

There is a licence inconsistency in the repository, and it is the kind of thing to resolve before you depend on the data.

The licence field of the repository lists CC0-1.0, which is a public domain dedication with no attribution requirement. The README's own licence section, however, renders a Creative Commons Attribution 4.0 International Licence badge and states that disclose is licensed under the Creative Commons Attribution 4.0 International Licence. Those are different terms. CC0 asks nothing of you, CC BY 4.0 requires attribution. The LICENSE file at the root of the repository is the authoritative text, and the repository is the one to ask if the two statements remain in conflict. This is a pointer to where to check, not legal advice.

The second limitation is inherent to the model. Every entry is a pointer to a programme that lives somewhere else, and those programmes change their scope, their safe harbor terms and their contact details on their own schedules. The dataset records what someone submitted at some point, and the README's own framing of the directory as the always-current place to look programs up is a tacit acknowledgement that the file in git is not the live truth.

The third is coverage of the long tail. The contribution route is a pull request, which means the set of programmes depends on people who care enough to file one. Large programmes are well represented for the obvious reason. Small programmes, regional ones and ones that have recently changed policy are the ones most likely to be missing or stale.

The fourth is that the safe harbor status is a summary. A field recording whether a programme has a safe harbor commitment tells you that it does, not what the commitment covers, and the programme's own text is what governs. The repository is a pointer to that text, not a substitute for reading it.

Editorial conclusion

Adopt diodb if you need a machine-readable list of vulnerability disclosure programs and their safe harbor status, since the whole dataset is one JSON file you can download and parse, and pull requests are the correction mechanism. Do not treat it as a live service, because the README is explicit that the directory at directory.disclose.io is the live front end and the repository is the dataset behind it, which means the file can lag. Verify first the licence, because the licence field of the repository lists CC0-1.0 while the README text names a Creative Commons Attribution 4.0 International Licence, and check program-list-schema.json against your consumer before you rely on field names.

Frequently asked questions

Official sources

1. disclose/diodb on GitHub
2. Issues
3. License: CC0-1.0
4. Project website
5. README