Open-Prompt-Injection: A Benchmark Harness for Prompt Injection Attacks and Defenses
This repository provides a benchmark for prompt injection attacks and defenses in LLMs
At a glance
- What is it?
- Open-Prompt-Injection is a Python toolkit from liu00222 that wires target tasks, injected tasks, attackers, defenses and evaluators into one measurable pipeline. It is a research instrument, not a runtime guardrail, and its factory-based design is both its strength and its friction.
- Who is it for?
- Adopt Open-Prompt-Injection if you need a repeatable way to measure attack success rate across attack, defense and task combinations, and you are willing to supply your own model API keys and checkpoints. Do not adopt it if you want a production filter sitting in front of a live LLM endpoint, because the repository is a benchmark and evaluation toolkit rather than a deployed guardrail.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 3 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The measurement gap Open-Prompt-Injection fills
Prompt injection research has a reproducibility problem. Papers describe an attack, a defense, and a number, but the surrounding apparatus (which target task, which injected task, which model, which prompt template) is often scattered across scripts that never get published. Open-Prompt-Injection addresses that by making the apparatus itself the deliverable. The README describes it as an open-source toolkit for prompt injection attacks and defenses that enables implementation, evaluation, and extension of attacks, defenses, and LLM-integrated applications and agents. The intended user is a researcher or security engineer who wants to compare two defenses on the same attack under the same task configuration, not someone who wants to paste a filter into a chatbot. The repository's own framing is instructive: it points to slides from a Safer with Google Summit 2025 presentation for background, which places the project in an academic-to-practitioner pipeline rather than a product pipeline. Nothing in the README claims production readiness, and that restraint is appropriate.
Tasks, attackers, apps and evaluators as composable objects
The architecture is a small object graph assembled through factory functions. PI.create_task loads a task config (for example ./configs/task_configs/sst2_config.json) and a sample count. PI.create_model loads a model config such as ./configs/model_configs/palm2_config.json. PI.create_attacker takes a strategy name and an injected task; the README shows 'combine' as one strategy. PI.create_app wraps a target task, a model and a defense name, where the demo passes defense='no'. PI.create_evaluator then takes the collected attack responses plus both tasks and exposes an asv attribute, the attack success value. The data flow is linear: iterate over the target app, call attacker.inject(data_prompt, i, target_task=target_task.task) to produce a contaminated prompt, query the app, and append the response. That loop is the whole benchmark in miniature. The design choice worth noting is that the injected task is created with for_injection=True, which is how the same task-config format serves two roles. It is a tidy abstraction, and it means swapping a defense is a string change rather than a code rewrite.
Getting the environment and a first query running
Installation assumes conda. The README gives two commands: conda env create -f environment.yml --name my_custom_env, then conda activate my_custom_env. Before any model call works you must edit ./configs/model_configs/palm2_config.json and replace the placeholder API keys, and the README directs you to Google's site for a PaLM2 key and to Meta's and OpenAI's sites for Llama and GPT registration. The minimal smoke test is a model query: build the config with open_config, call PI.create_model(config=model_config), print model info, and query with a plain string such as "Write a poem about monkeys". For the full experiment matrix the entry point is python3 run.py, with the matrix declared in configs/experiment_matrix.json. Two operational details in the README are easy to miss and both matter: run.py --dry-run validates paths and prints every command without starting model processes, and response archives carry a configuration fingerprint so they are reused only when the fingerprint matches the active experiment. Archives without metadata still load, but they produce a warning.
Detectors and localizers are separate, downloaded artifacts
The defense side of the toolkit is not self-contained. DataSentinel is used as a prompt injection detector, and its fine-tuned checkpoint is distributed through a Google Drive link rather than a package index. PromptLocate localizes injected prompts and recovers the original, and its LoRA adapter is likewise a Drive download. Both are wired through a model config where config["params"]['ft_path'] is set to the downloaded model location, and both examples use ./configs/model_configs/mistral_config.json as the base. The README then composes them into a pipeline: DataSentinel decides whether a prompt is contaminated, and if it is, PromptLocate localizes and recovers. That composition is the most practically interesting part of the repository, and it is also the part with the least documentation. The README states that more detectors and fine-tuning code will be released soon, which is an admission that the current set is partial. Anyone evaluating the defense pipeline is evaluating two checkpoints hosted outside the repository as much as they are evaluating the repository itself.
Where the toolkit stops being the right tool
Three constraints stand out. First, the benchmark measures attack success value under controlled task configurations; it does not measure latency, cost, or false-positive rate on benign traffic, and the README does not document any of those. A defense that scores well on ASV can still be unusable if it flags ordinary user input, and nothing here tells you that. Second, the checkpoints for DataSentinel and PromptLocate arrive via Google Drive, so reproducibility depends on links that the project does not control; if a link rots, the defense half of the pipeline becomes unrunnable while the attack half still works. Third, the model layer leans on hosted APIs. The demo config is PaLM2, and the detector examples use Mistral configs with a local fine-tuned path, so a fully offline run depends on which model configs you supply. The README does not document rollback, rate-limit handling, or what happens when a provider changes an API. For a research harness that is tolerable; for anything with an uptime expectation it is disqualifying.
How it differs from Garak and similar scanners
Garak, the LLM vulnerability scanner from NVIDIA, takes a different approach: it ships a library of probes and detectors and runs them against a target endpoint, reporting which probes elicited which failures. Open-Prompt-Injection instead treats the attack as a transformation over a labeled dataset. The injected task has ground-truth labels, the target task has ground-truth labels, and the evaluator computes ASV from the model's responses. That means the output is a rate over a defined sample, not a probe-by-probe report. The practical difference: Garak answers "what breaks this endpoint", while Open-Prompt-Injection answers "how often does this attack succeed against this task pair under this defense". If you need a number you can put in a paper or a comparison table, the factory-and-evaluator design is the better fit. If you need broad coverage of known failure modes against a live service, Garak's probe library is broader, and Open-Prompt-Injection's two named detectors will feel narrow by comparison.
Maintenance, licence and the cost of staying current
The repository is not archived, and the last push was on 2026-09-12, one day before this writing. The licence is MIT, which permits commercial use and modification provided the copyright notice and permission notice are retained; that is a permissive arrangement, though as always the licence text governs and this is not legal advice. The upgrade cost is concentrated in two places. The model configs are provider-specific, so an API change at Google, Meta or OpenAI can break a config without any change in this repository. The checkpoints are external downloads, so a new detector release means re-downloading weights and re-pointing ft_path. The run.py archive fingerprinting helps here: because archives are reused only when the configuration fingerprint matches, changing a config will not silently mix old and new results, though the README notes that historical archives without metadata remain readable and produce a warning. That warning is the thing to watch if you keep results across versions.
Editorial conclusion
Adopt Open-Prompt-Injection if you need a repeatable way to measure attack success rate across attack, defense and task combinations, and you are willing to supply your own model API keys and checkpoints. Do not adopt it if you want a production filter sitting in front of a live LLM endpoint, because the repository is a benchmark and evaluation toolkit rather than a deployed guardrail. Before committing, verify that the model config files you intend to use match providers you can actually authenticate against, and check whether the DataSentinel and PromptLocate checkpoints download correctly, since those two components carry the repository's defense story.
Community notes