# OWASP AI Testing Guide: a methodology you read, not a tool you run

> The OWASP AI Testing Guide is a community-written methodology for testing AI systems, delivered as a document site rather than as software. It suits practitioners who need a testing vocabulary and a reading list, and it is the wrong choice for anyone expecting automated checks against a live model.

**OWASP/www-project-ai-testing-guide** — OWASP Foundation web repository

- Repository: https://github.com/OWASP/www-project-ai-testing-guide
- Stars: 831 · Forks: 159
- Language: Python
- License: NOASSERTION
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/owasp-www-project-ai-testing-guide

## A guide that ships as documents, not as an installable package

The OWASP AI Testing Guide is easy to mistake for a testing tool, so it is worth being clear about what the repository actually contains. The project describes itself as an open-source initiative aimed at providing structured methodologies and best practices for testing artificial intelligence systems. What sits in the tree backs that up. There is a Document directory that holds the guide text, a PDFGenerator directory, a Gemfile, a _config.yml file, index.md, info.md, leaders.md and a project.owasp.yaml file. There is no test harness, no command line entry point and no library package. The primary language tag is Python, but nothing in the listed top-level entries is an importable module.

That distinction drives everything else about how you would use the project. A tool that ingests a model endpoint and returns a pass or fail verdict can be adopted without changing how a team already works. This guide cannot. It is a document that has to be read, argued with, and turned into test cases that someone on your team then executes by hand or by whatever automation you already maintain. The stated audience is practitioners, researchers and organisations that want a shared frame for AI testing, which is a reasonable goal when teams disagree about what counts as a proper AI test at all.

## The entry point is a table of contents, not a getting started script

The single navigational instruction in the top-level README is a link to the AI Testing Guide Table of Contents, pointing at Document/README.md. That is the whole of the onboarding path. There is no installation command, no container image, no pip package, no configuration file to copy and no first run to observe. For a reader who arrives expecting a tool, the absence is jarring. For a reader who arrives expecting a methodology, the table of contents is exactly the right entry point, because it is the skeleton of the test plan and it tells you in one screen how the methodology is decomposed.

The surrounding files support the same reading. A Gemfile and a _config.yml sit beside the content, which is the conventional arrangement for a Jekyll style static site, and PDFGenerator indicates that a built artefact is produced for people who want the guide offline or as a handout. The project also carries a SECURITY.md file, tab_contributing.md and tab_roadmap.md, plus a 404.html page and an assets directory holding an image referenced from the README. These are the parts of a project that expects to be read in a browser and printed, not built against. There is no tag, no release, and the GitHub releases feed is empty, so there is no versioned artefact to pin your internal process to either.

## The guide positions itself between six neighbouring frameworks

Rather than inventing a taxonomy from scratch, the project leans on work that already exists and lists those dependencies openly in its README. The OWASP GenAI Red Teaming Guide is cited as a structured, risk-based methodology spanning model evaluation and system integration pitfalls, with attention to runtime behaviour. The Cloud Security Alliance Agentic AI Red Teaming Guide covers attack surfaces specific to autonomous and goal-driven agents. The OWASP AI Exchange is described as contributing to international standards with more than 200 pages of practical advice on protecting AI and data centric systems. The OWASP AI Security and Privacy Guide is a working document covering design, creation, testing and procurement. The OWASP Top 10 for LLM names the leading risks across development, deployment and management, and the OWASP AI VSS adapts CVSS style principles into a scoring framework for AI specific weaknesses.

The NIST AI 100-2 E2025 adversarial machine learning taxonomy is listed as well. That is a useful map for anyone deciding where this project fits, and it also sets a fair expectation. The stated objective is consolidation rather than novelty: to take knowledge from these sources and shape it into one framework for testing AI systems. If you already run a full red teaming programme under one of the cited guides, this project is a crosswalk to that ecosystem, not a replacement for it.

## Writing your first test case from the guide

Because there is no command to run, the first practical use is a reading and translation exercise, and the source for that exercise is Document/README.md. Read it as a table of test objectives, then pick the objectives that map to a system you actually own. A team shipping a retrieval augmented assistant would select the objectives that touch retrieval quality, prompt handling and system integration, and leave the ones that concern training pipeline behaviour unmarked.

From there, the work is entirely yours, and it is worth writing the resulting cases in the format your organisation already uses rather than inventing a parallel one. A test case needs an identifier, the system component in scope, the data that goes in, the behaviour you expect, and the evidence you will keep when it fails. The guide gives you the ordering and the framing of those categories, which is genuinely useful, since the hard part of AI testing is deciding what to assert against a system whose output is not deterministic. The README does not define assertion formats, tolerances for probabilistic outputs, or a required evidence format, so those decisions stay with the reader. Treat the table of contents as a prompt for scoping a test plan, not as a specification you can hand to an engineer unchanged.

## Where this project stops and other tools begin

The honest limitation is that nothing here automates a single check. There is no fuzzing harness, no adversarial sample generator, no model evaluator, no prompt injection scanner and no scoring script in the top-level tree. The Python language tag reflects a small amount of tooling inside the project, not a library API. Teams who need continuous verification against a live endpoint will find no package to install and no CI target to invoke.

The same limitation applies to scoring. The OWASP AI VSS project is a separate resource with its own site, and the guide links out to it rather than shipping a calculator. If your requirement is a repeatable severity number for an AI finding, you need that scoring system and a scoring tool, and this repository supplies the narrative, not the number. There is also no documentation here on model drift monitoring, regression thresholds or environment configuration, since there is no environment to configure. The guide is best treated as a planning document that sits upstream of tooling, and its value depends entirely on whether you are willing to convert prose objectives into executable checks with the resources you already have.

## Contributing goes through two named project leaders

The collaboration path is unusually concrete for a document project. The README directs contributors, feedback and collaborators to two named project leaders, Matteo Meucci and Marco Morana, with a contact address for each. A CONTRIBUTING.md file and a tab_contributing.md page are present in the tree, alongside tab_roadmap.md for direction and project.owasp.yaml for the project metadata that OWASP uses to list chapters. The repository is not archived, so pull requests are a realistic route for adding or correcting a test objective.

The trade-off is that the structure is the OWASP chapter structure rather than a software project structure. There is no issue template, no release train, no semantic versioning of the document and no published changelog beyond the GitHub commit history. If you contribute, expect your material to be merged into a prose hierarchy that other people are also editing, and expect the same text to change shape as different contributors align wording across chapters. Review cycles are driven by volunteer maintainers, which is a realistic way to run a standards document and a slow way to ship one.

## Maintenance rests on a single push date and a custom licence

The repository is not archived, and the last push was on 2026-06-01. There are no GitHub releases, so there is no numbered version to cite and no upgrade path to walk. This is consistent with what the project is: a document that evolves through commits on the default branch, main. Because no version is published, teams that depend on this methodology should track the main branch or mirror the table of contents internally rather than expect a stable release to pin.

Licensing deserves attention before you copy content. The repository is listed under a custom licence that GitHub cannot classify, and the full terms live in the LICENSE.md file. The top-level README states no grant and no attribution rule in readable form, so reuse of the text, including in internal test plans or commercial documentation, needs to be checked against that file directly. The OWASP Foundation name appearing in the description and in the leader contact addresses does not by itself settle what you may do with the content. Read LICENSE.md, then mirror the structure with attribution if that is what it requires.

## Conclusion

Adopt this guide if you are writing your own AI test plan and want a shared vocabulary, a defined table of contents and a curated pointer to the OWASP GenAI Red Teaming Guide, the OWASP Top 10 for LLM and the OWASP AI VSS scoring model. Do not adopt it if you need automated adversarial test generation, prompt fuzzing or a regression suite against a deployed endpoint, because nothing in the repository tree provides runnable tooling for that. Before committing, open Document/README.md and confirm that the test case structure it defines matches the way your organisation already writes test plans, and check the LICENSE.md file, since the project carries a custom licence that GitHub cannot classify and no permissive terms are stated in the top-level README.

## FAQ

### Is the OWASP AI Testing Guide a tool I can install and run against a model?

No. The repository ships a Document directory, a PDFGenerator directory and site files such as a Gemfile and a _config.yml, and the README sends readers to Document/README.md for the table of contents. No install command, package name or runner is documented.

### How does the OWASP AI Testing Guide differ from the OWASP GenAI Red Teaming Guide?

The README lists the GenAI Red Teaming Guide as an existing initiative with its own risk-based methodology for model evaluation and runtime behaviour, and positions the AI Testing Guide as an effort that consolidates such work into one testing methodology. The two are related but separate projects.

### How do I get a severity score for an AI vulnerability found with this guide?

The guide links to the OWASP AI VSS scoring system, a separate project that adapts CVSS principles to AI specific vulnerabilities. The guide itself contains no scoring calculator or severity output.

### How active is the project and can I expect releases?

The repository is not archived, and the last push was on 2026-06-01. The releases feed is empty, so there are no published versions to upgrade to and work happens on the main branch.

## Sources

- [Issues](https://github.com/OWASP/www-project-ai-testing-guide/issues)
- [OWASP/www-project-ai-testing-guide on GitHub](https://github.com/OWASP/www-project-ai-testing-guide)
- [README](https://github.com/OWASP/www-project-ai-testing-guide/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/owasp-www-project-ai-testing-guide