PurpleLlama: Meta's Umbrella Repo for LLM Safety Tooling

What PurpleLlama actually is, and who it is for

PurpleLlama is described in its README as an umbrella project that will bring together tools and evals for building responsibly with open generative AI models. The initial release covers two areas: cybersecurity evaluation and input/output safeguards. The name comes from the security industry's purple teaming concept, where red team (attack) and blue team (defence) postures are combined. Meta states that the same ethos applies here. The practical audience is developers running Llama models who need moderation or evaluation components they can host themselves, rather than an API-based safety service. The repository is Python, on the main branch, and the README lists no releases, so consumers should expect to track the branch rather than pin to a tagged version.

The four components and what each one filters

Llama Guard 3 is a series of input and output moderation models, fine-tuned from Meta-Llama 3.1 and 3.2, built around the MLCommons hazards taxonomy. The README states they support a 128k context window, image reasoning, and seven additional languages. Prompt Guard targets prompt attacks, split into two categories: prompt injections, which exploit untrusted third-party data placed in the context window, and jailbreaks, which are instructions designed to override a model's built-in safety behaviour. Code Shield filters insecure code at inference time, and the README lists three uses: mitigating insecure code suggestions, preventing code interpreter abuse, and secure command execution. CyberSec Eval is the benchmark side. Version 1 measured insecure code suggestions and compliance with malicious requests; version 2 adds code interpreter abuse, offensive cybersecurity capability, and prompt injection susceptibility. These are four different mechanisms solving four different problems, and treating the repo as one product will lead to wrong integration decisions.

How the safeguards sit in the request path

The README frames the safeguards as system-level checks: it recommends that all inputs and outputs to the LLM be checked and filtered according to content guidelines appropriate to the application, referencing Llama 3's Responsible Use Guide. That places Llama Guard on both sides of the model call, as an input classifier and an output classifier, with Prompt Guard focused on the input side where untrusted data enters the context window. Code Shield operates at inference time on generated code, which means it inspects output before that output reaches an execution environment, such as a code interpreter. The README does not include a reference architecture diagram, a gateway implementation, or an orchestration layer. What it provides is the classifiers and the benchmark harnesses; the placement, batching, thresholds, and failure handling are left to the integrator. That is the main architectural fact to absorb before planning an integration.

Getting the components running: what the material supports

The repository material does not include a top-level install command, a requirements file listing, or a CLI entry point, so any claim about a single pip install for the whole project would be invented. What the README does give is a concrete starting point for one component: a CodeShield example notebook at CodeShield/notebook/CodeShieldUsageDemo.ipynb, described as the CodeShield usage demo. That notebook is the documented path for seeing Code Shield in action. For the Llama Guard and Prompt Guard models, the README points to Meta's Hugging Face organisation, and the licence table links each checkpoint family to its own licence file, including LICENSE in this repository for Llama Guard 3-8B, Llama Guard 3-1B, Llama Guard 3-11B-vision, and Prompt Guard. Treat those links as the authoritative source for which terms attach to which weights. Beyond that, the README does not document configuration keys or environment variables, and I will not guess at them.

The licence split is the first thing to check, not the last

This is where the repository layout matters more than the marketing. The README's licence table separates component types. Evals and benchmarks, currently the Cyber Security Eval, are MIT. The safeguard models are not: Llama Guard 1 uses the Llama 2 Community License, Llama Guard 2 uses the Llama 3 Community License, and Llama Guard 3-8B, Llama Guard 3-1B, Llama Guard 3-11B-vision, and Prompt Guard all use the Llama 3.2 Community License. Code Shield is listed as MIT. The repository itself carries a NOASSERTION licence identifier, which is consistent with a project that has no single licence. The practical consequence is that a team can ship the MIT-licensed benchmark code and Code Shield under permissive terms, but the moderation models carry the Llama Community licence conditions, which typically include redistribution and acceptable-use obligations. I am not giving legal advice; the point is that the licence question has to be answered per component, and the README's table is the map for doing that.

Where the approach breaks down

The safeguards are Llama-specific. Llama Guard 3 is fine-tuned from Meta-Llama 3.1 and 3.2, and the licence terms attach to those model families. A team running a mixed fleet of models, or one that needs a single moderation policy applied uniformly across vendors, will end up maintaining parallel systems rather than one. The second limitation is scope. The README frames the safeguards as content moderation and prompt-attack detection; it does not claim to cover data exfiltration, tool-use authorisation, or agentic action boundaries. Prompt Guard addresses injections and jailbreaks, which is a narrow slice of the broader prompt-security problem. Third, the repository is presented as an umbrella that will bring together more tools over time. That is a statement about future contents, not a current guarantee, and it means the component set you adopt today may not be the component set the project converges on. Finally, the absence of releases in the material means there is no versioned artefact to pin against, which complicates reproducibility for anyone who needs a frozen dependency set.

The realistic alternative, and where it differs

The closest alternative in approach is a hosted moderation API, such as a cloud provider's content-safety endpoint. The difference is not quality, it is where the decision runs. A hosted endpoint keeps the classifier off your infrastructure: you send text out, you get a label back, and you inherit the provider's update cadence and its policy taxonomy. PurpleLlama inverts that. The models run in your environment, the taxonomy is the MLCommons hazards set rather than a vendor's evolving policy, and the licence obligations move from a service agreement to the Llama Community License. That trade is favourable when you cannot send user content to a third party, when you need to inspect or modify the classifier, or when you want the same moderation logic in an air-gapped or on-premise deployment. It is unfavourable when you have no GPU capacity to spare for a separate moderation model, when you need coverage across non-Llama models under one policy, or when you want someone else to own taxonomy updates. The benchmark side has a different comparison: CyberSec Eval is a measurement suite, and its value depends on whether the specific risks it measures (insecure code suggestions, interpreter abuse, offensive capability, prompt injection) match the risks you care about. A benchmark that does not measure your failure mode is not evidence about your system.

Maintenance cost and what to verify before committing

The maintenance picture follows from the structure. Four components with three licence regimes means three sets of terms to track, and the Llama Community licences are versioned by model family, so a future Llama Guard built on a newer base model may arrive under a different licence than the one you adopted. The project describes itself as one that will bring together more tools over time, so the surface area is likely to grow, and each addition is another component to evaluate. The absence of releases means upgrades are branch-tracking exercises unless the maintainers begin tagging. Before committing, verify three things against the material in this repository: which licence file is linked for the exact Llama Guard or Prompt Guard checkpoint you intend to deploy, whether the CodeShield notebook at CodeShield/notebook/CodeShieldUsageDemo.ipynb covers the filtering path your application needs, and whether the CyberSec Eval benchmark version whose risks you care about is the one you can actually run against your model. Everything else about integration is on you, because the README stops at describing the components.

Editorial conclusion

PurpleLlama fits teams already serving Llama models that want moderation, prompt-attack detection and insecure-code filtering as separate, self-hosted components. It does not fit teams that need a single policy engine across multiple model vendors, or that cannot accept the Llama 3.2 Community License terms on the safeguard models. Before adopting, verify the exact licence file attached to the specific Llama Guard checkpoint you plan to ship, and check whether the Code Shield path you need is covered by the notebook example or requires wiring the filter into your own inference loop.

Official sources

1. Issues
2. meta-llama/PurpleLlama on GitHub
3. README