DeepTeam: an open source red teaming harness for LLM systems and agents
DeepTeam is a framework to red team LLMs and AI agents.
At a glance
- What is it?
- DeepTeam wraps 50+ vulnerability definitions and a set of simulated attacks into a Python framework that runs locally on top of DeepEval. It is useful when you need repeatable, scored red team runs in CI, and less useful when you need adversarial creativity or a managed risk register.
- Who is it for?
- Adopt DeepTeam if your team already writes Python tests and you want red team runs to produce binary pass/fail scores you can gate a pipeline on. Do not adopt it if you need a managed risk register, cross-run dashboards, or a human red team's improvisation; the framework's own README points at the Confident AI platform for storing and sharing results, which tells you where that boundary sits.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 26 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What problem DeepTeam is aimed at
Most teams shipping an LLM feature have no repeatable way to ask whether it can be talked into leaking a system prompt, executing a shell command, or returning biased output. Manual probing happens once, before launch, and then never again. DeepTeam's pitch is that this should look like penetration testing: a defined set of vulnerability categories, a set of attacks that try to trigger them, and a scored result you can compare across runs. The README describes it as "a simple-to-use, open-source red teaming framework for LLM systems" and frames the target as AI agents, RAG pipelines and chatbots. The audience is therefore engineers who own an LLM-backed service and want the security check to live in the same repository as the code, not in a spreadsheet. The project is Apache-2.0, written in Python, and the README states it runs locally on your machine. That local execution matters: the vulnerability metrics are LLM-as-a-judge calls made from your process, so your prompts and your model's responses go to whichever judge model you configure, not to a vendor endpoint by default.
Vulnerability metrics and the judge loop
The core mechanism is a catalogue of over 50 vulnerability definitions, grouped in the README into Data Privacy, Responsible AI, Security, Safety and Business. Each entry maps to a named risk with its own documentation page: PII Leakage and Prompt Leakage under Data Privacy; Bias, Toxicity, Child Protection, Ethics and Fairness under Responsible AI; BFLA, BOLA, RBAC, Debug Access, Shell Injection, SQL Injection, SSRF, Tool Metadata Poisoning, Cross-Context Retrieval and System Reconnaissance under Security. The README states that each vulnerability uses LLM-as-a-Judge metrics that run locally to produce binary pass/fail scores with reasoning. That design choice is the whole architecture in one sentence: an attack generates an input, your target system answers, and a judge model grades the exchange against the vulnerability's criteria. It also means the judge model is part of your test harness. Swap it and your pass rates can move without a single change to the system under test, which is the first thing to pin down before you trust a score. The Security list is the most interesting part of the catalogue, because entries like BOLA, BFLA and Cross-Context Retrieval only make sense against agents that call tools and read from a retrieval layer. A plain chatbot has nothing to attack there.
Attacks: jailbreaking, injection and multi-turn exploitation
Vulnerabilities describe what can go wrong. Attacks are how DeepTeam tries to make it happen. The README names jailbreaking, prompt injection and multi-turn exploitation among the simulated attacks. Multi-turn is the one worth pausing on, because it implies the harness holds a conversation state across turns rather than firing single prompts, which is closer to how a real user erodes a refusal. The v0.1.9 release notes also list "Agentic Red Teaming" as a feature, which lines up with the Security vulnerability categories: attacking an agent means attacking its tool calls and its retrieval boundaries, not just its text output. What the README does not give is a per-attack breakdown of how each one is constructed, how many variants it generates, or how the attack set is sampled for a given vulnerability. Those details live on the documentation site rather than in the repository README, so budget time to read them before you assume a given attack maps to the threat model you care about.
Getting a run started
The README's Getting Started section is linked as a quickstart anchor rather than reproduced, so the concrete commands live on trydeepteam.com. What the repository does establish is the shape of the setup: a Python package built on DeepEval, a CLI introduced in v0.1.9, and a judge model you supply. Three things are worth verifying from the docs before you write anything. First, the exact package name and install command, since the README does not print it inline. Second, how a target system is registered: DeepTeam has to call your agent or chatbot somehow, and that adapter is where most integration effort goes. Third, how the judge model is configured, because that key determines both your score stability and your API spend. The CLI is the lighter path if you want a one-off assessment; the Python API is the one that belongs in a test suite. Version numbers matter here: v1.0.9 is labelled the first stable release and is dated 2025-11-12, while v0.1.4 also carries a "First Stable Release" label from 2025-05-23. Two releases claiming the same milestone is a signal that the project's own versioning story shifted, so pin an exact version in your requirements file rather than tracking the default branch.
Where DeepTeam stops being the right tool
The judge model is the sharpest limitation. Binary pass/fail from an LLM judge is convenient for gating a pipeline and unreliable as a measurement instrument. The same attack against the same response can flip depending on the judge, the temperature, and the surrounding prompt. DeepTeam does not remove that problem; it packages it. If your team needs a defensible number for a compliance document, an LLM judge's reasoning string is not the same artefact as a human assessor's finding. The second limitation is coverage. A catalogue of 50+ named vulnerabilities is a starting taxonomy, not a threat model for your specific product. Prompt injection against a RAG pipeline that reads from a shared vector store is a different problem from prompt injection against a customer-facing chatbot, and the catalogue entry will not know which one you have. Third, the README itself flags that results need somewhere to live and points to the Confident AI platform for managing risk assessments, monitoring vulnerabilities in production and sharing reports. That is a fair statement of scope, and it is also an admission that the open source framework produces local results and stops there. If your process requires tracked findings with owners and remediation dates, DeepTeam is one half of that process.
How this differs from general LLM evaluation frameworks
DeepEval, which DeepTeam is built on, evaluates whether an LLM application is correct: answer relevancy, faithfulness to retrieved context, hallucination. Its orientation is quality. DeepTeam inverts the question and asks whether the application can be made to fail in a specific, named way. The practical difference shows up in the test cases. An evaluation suite feeds the system realistic user inputs and checks the output against expected behaviour. A red team suite feeds the system adversarial inputs designed to break a rule, and treats a pass as the absence of a finding rather than the presence of quality. That inversion has a cost: red team runs are only as good as the attacks, and a small attack set produces false confidence. A general evaluation framework with a custom metric could be bent into red teaming, and teams do that, but you would be writing the vulnerability definitions and the attack generators yourself. DeepTeam's contribution is the pre-built catalogue and the attack simulation layer around it, not the scoring machinery, which it inherits.
Guardrails, maintenance and licence
The README mentions that DeepTeam "also offers guardrails to prevent these issues in production." That is a single sentence in the introduction, with no detail in the material available here on how a guardrail is configured, whether it runs as middleware, or how it relates to the vulnerability metrics. Treat it as an unverified claim until you read the guardrail documentation, and do not adopt the framework for its runtime guardrails alone on the strength of that line. Maintenance cost is the other open question. The repository shows a push in August 2026 and a stable release in November 2025, so the project is active, but the release history is short and the version numbering has already been reset once. Expect to re-pin versions rather than float. The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant; it does not impose copyleft on your own code. That said, this is a description of the licence identifier, not legal advice, and if you are embedding DeepTeam into a product you ship, have your own counsel read the LICENSE.md file in the repository rather than a review like this one.
Editorial conclusion
Adopt DeepTeam if your team already writes Python tests and you want red team runs to produce binary pass/fail scores you can gate a pipeline on. Do not adopt it if you need a managed risk register, cross-run dashboards, or a human red team's improvisation; the framework's own README points at the Confident AI platform for storing and sharing results, which tells you where that boundary sits. Before committing, verify three things in your own checkout: that the judge model you configure produces stable scores across repeated runs of the same attack, that the vulnerability list covers the agent surfaces you actually expose (tool schemas, retrieval boundaries, authorization paths), and that the Apache-2.0 licence file in the repository matches what your legal review expects for a dependency that will sit in your test suite.
Community notes