Purple Llama splits LLM safety into moderation models and attack benchmarks
Set of tools to assess and improve LLM security.
At a glance
- What is it?
- Purple Llama is Meta's umbrella repository holding Llama Guard, Prompt Guard, Code Shield and the CyberSec Eval benchmark suites. The useful part is the split between runtime filters and offline measurement, and the confusion comes from how many model families live under one roof.
- Who is it for?
- Adopt Purple Llama if you are shipping an open model and need both a runtime content filter and a repeatable cyber risk measurement, since the safeguards and the benchmarks live in the same place and share vocabulary. Do not adopt it as a single installable tool, and do not treat CyberSec Eval scores as a ship gate the project will enforce for you.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- meta-llama/PurpleLlama4,480 ★ · +89
| Repository | 2026-09-13 | 2026-11-15 |
|---|---|---|
| meta-llama/PurpleLlama | 4.4k | 4.5k |
89 Stars in the last 63 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
One repository, two different jobs: filtering traffic and measuring risk
Purple Llama is a container, not a tool. The README describes it as an umbrella project that will gather tools and evals for building responsibly with open generative AI models, and the first wave covers cybersecurity evaluation plus input and output safeguards. That framing matters for anyone deciding what to adopt, because the two halves answer different questions. Llama Guard, Prompt Guard and Code Shield sit in your request path and return a verdict on content. CyberSec Eval v1 and v2 sit outside your application and score a model on a benchmark. If you only need the second, none of the safeguard directories apply to you. If you need only the first, the benchmark suites are dead weight. The top level of the repository makes the split visible: CodeShield/, CybersecurityBenchmarks/, Llama-Guard/, Llama-Guard2/, Llama-Guard3/, Llama-Guard4/, Llama-Prompt-Guard-2/, LlamaFirewall/, Prompt-Guard/ and SensitiveDocClassification/ each sit side by side rather than under a shared package.
Llama Guard 3 is tuned to the MLCommons hazard taxonomy
The moderation path runs through Llama Guard 3, a series of high performance input and output moderation models built by fine tuning Meta-Llama 3.1 and 3.2. The README states they were optimized to detect the MLCommons standard hazards taxonomy, which is the detail that separates this from a generic toxicity classifier: the categories a Guard 3 verdict returns are drawn from a named external standard rather than from a list the project invented. The same paragraph adds that the models support the Llama 3.2 capabilities including 7 new languages, a 128k context window and image reasoning. A 128k window matters operationally, since a long context is exactly where an injected instruction can hide several thousand tokens from a human reviewer. The stated purpose is narrower than safety in general. These are content classifiers for inputs and outputs, and the README itself ties the recommendation to Llama 3's Responsible Use Guide, which asks that all inputs and outputs be checked against guidelines appropriate to the application.
Prompt Guard watches the context window, not the answer
Prompt Guard sits upstream and targets two attack shapes the README defines precisely. A prompt injection exploits untrusted data from third parties that has entered the model's context window in order to get the model to execute unintended instructions. A jailbreak is a malicious instruction built to override the safety and security features inside a model. Both are about the request, not the response, which is the practical difference from Llama Guard's output moderation. The failure mode follows from that position. Prompt Guard has to be in front of the model and ahead of any retrieval step that puts external text in the window, otherwise there is nothing for it to inspect. A pipeline that calls a guarded model but hands it unfiltered web or document text has not closed the injection path, whatever its output filter reports.
Code Shield filters insecure code at inference time
Code Shield extends the same idea to source code. The README positions it as inference time filtering of insecure code produced by LLMs, with three named jobs: mitigating insecure code suggestions, preventing code interpreter abuse and supporting secure command execution. The third job is the interesting one, because it targets the case where a model writes commands that a host then runs. A text classifier that only inspects the reply can flag a dangerous string and still miss a syntactically valid but destructive command line. Llama Guard 3 is also described as optimized to detect helpful cyberattack responses and for preventing malicious code output from reaching hosting environments that run code interpreters, so the two components overlap on purpose. The usage material for Code Shield is a notebook linked from the README at CodeShield/notebook/CodeShieldUsageDemo.ipynb.
CyberSec Eval measures two things, and v2 widened the second
CyberSec Eval v1 was built on industry guidance such as CWE and MITRE ATT&CK and aimed at three outputs: metrics for quantifying LLM cybersecurity risk, tools for evaluating the frequency of insecure code suggestions, and tools for evaluating how hard a model is to use for a cyber adversary. CyberSec Eval 2 expands on its predecessor by measuring propensity to abuse a code interpreter, offensive cybersecurity capability, and susceptibility to prompt injection. The shift matters for anyone who already ran v1. Code interpreter abuse and prompt injection susceptibility are application surface risks, so a model that scores well on v1's insecure code metric can still score badly under v2 if your product lets users upload data or run generated code. The README links a Hugging Face leaderboard for results. The MITRE ATT&CK reference also tells you what the benchmark considers an attack, which is the piece to read before quoting a score.
There is no install command, and the directory list runs ahead of the README
The README links models on Hugging Face and each component lives in its own directory, so an installation path means choosing which component you want and following that directory's own documentation. That is a real cost of the umbrella layout. The more striking gap is between the README and the tree. The README's licence table names Llama Guard, Llama Guard 2, Llama Guard 3-8B, Llama Guard 3-1B, Llama Guard 3-11B-vision, Prompt Guard and Code Shield. The repository also contains Llama-Guard4/, Llama-Prompt-Guard-2/, LlamaFirewall/ and SensitiveDocClassification/, none of which appear in the table or in the sections above it. Licensing is not uniform either: evals and benchmarks are MIT, Code Shield is MIT, while each guard model carries the community licence matching its Llama generation. Read the component directory before you assume the umbrella grants you one permissive set of terms.
Where Purple Llama stops and your own harness starts
The honest limitation is that a benchmark suite and a set of classifiers leave the hardest work to you. Purple Llama tells you a model can be scored on CyberSec Eval 2; deciding which score blocks a launch is up to you. Judge it as measurement scaffolding rather than a policy engine. A different class of approach is a general guardrails server or a hosted safety API, which give you per request configuration, latency accounting and a support commitment in exchange for routing content through another service and giving up the ability to run the classifier on your own inference hardware. Purple Llama's approach is the opposite: the model weights and the benchmark code are yours to host and to read, and the work of wiring them into a pipeline is entirely yours. Teams with their own inference stack and a security reviewer can absorb that. Teams without one will spend their time on plumbing before they reach a single verdict.
Editorial conclusion
Adopt Purple Llama if you are shipping an open model and need both a runtime content filter and a repeatable cyber risk measurement, since the safeguards and the benchmarks live in the same place and share vocabulary. Do not adopt it as a single installable tool, and do not treat CyberSec Eval scores as a ship gate the project will enforce for you. Verify first that the guard directory you pick matches the Llama version you actually deploy, because the model rows and the directory list have already drifted apart in this tree.
Frequently asked questions
What is Purple Llama?
It is an umbrella project collecting tools and evals for building responsibly with open generative AI models, covering cybersecurity evaluation and input and output safeguards.
Is Purple Llama a single installable tool?
The repository is a container rather than one installable tool, holding separate directories such as Llama-Guard/, Prompt-Guard/, CodeShield/ and CybersecurityBenchmarks/ that you pick between.
What is the Llama Guard Model?
Llama Guard 3 is a series of high performance input and output moderation models built by fine tuning Meta-Llama 3.1 and 3.2, optimized to detect the MLCommons standard hazards taxonomy.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/meta-llama-purplellama)Star history chart for your README
[](https://hysenlabs.com/projects/meta-llama-purplellama)Was this useful?