garak: an LLM vulnerability scanner modeled on nmap

What garak is actually for

garak describes itself as a Generative AI Red-teaming and Assessment Kit, and the README frames the goal narrowly: checking whether an LLM can be made to fail in a way we do not want. The listed targets are hallucination, data leakage, prompt injection, misinformation, toxicity generation and jailbreaks. The comparison the project makes for itself is to nmap and Metasploit Framework, which is a useful anchor. Those tools do not tell you whether a host is secure. They enumerate what responds and what it responds to, and the judgement stays with the operator. garak works the same way. It is aimed at people who already have a model in front of them and want a repeatable way to try known attack patterns against it, rather than at people looking for a certification or a single risk score. That framing also sets the audience: security engineers, ML platform teams and researchers who are comfortable reading a failure rate per probe rather than a headline number. The README states that garak combines static, dynamic and adaptive probes, which is a hint that not every probe is a fixed list of strings.

Probes, detectors and the generation budget

The mechanism is a pairing. Each probe is a family of attempts designed to elicit a specific failure, and each probe carries a recommendation about which detectors should judge the output. When you run garak without narrowing the scope, it tries all the probes it knows against the target and applies those recommended detectors. Output is reported per probe and per detector as a row, and any prompt attempt that produced an undesirable behaviour is marked FAIL with a failure rate attached. The README gives the row format as two numbers, for example 840/840, where the first is the total number of text generations and the second is how many behaved acceptably. That second number is where a lot of the interpretation lives. The README states that more than one generation is made per prompt, ten by default, so a probe with 84 prompts produces 840 generations. This matters for cost and for statistics. Ten samples per prompt is enough to notice a consistent failure and not enough to characterise a low-probability one, and every one of those generations is a real request to your target, which on a hosted API is a real bill. The README also notes that errors go to garak.log while the run itself is logged in detail to a .jsonl file specified at analysis start and end. The README text is truncated where it begins to describe a basic analysis script, so what that script does is not something I can confirm from the supplied material.

Installing garak and pointing it at a model

The standard install is a single pip command: python -m pip install -U garak. The README notes that the PyPI version is updated periodically, and offers python -m pip install -U git+https://github.com/NVIDIA/garak.git@main for a fresher build. For source work it gives a Conda recipe: conda create --name garak "python>=3.11,<=3.13", then conda activate garak, gh repo clone NVIDIA/garak, cd garak, and python -m pip install -e . The Python range is stated explicitly, so a 3.10 or 3.14 interpreter is outside what the README supports. There is also a note for anyone who cloned before the move to the NVIDIA organisation: git remote set-url origin https://github.com/NVIDIA/garak.git. Running a scan needs a target type and, optionally, a target name. The README's own example for a commercial model is export OPENAI_API_KEY="sk-123XXXXXXXXXXXX" followed by python3 -m garak --target_type openai --target_name gpt-5-nano --spec probes.encoding. For a local Hugging Face model it gives python3 -m garak --target_type huggingface --target_name gpt2 --spec probes.dan.Dan_11_0. The README states that some generators need an API key in an environment variable and will say so when they do. Supported backends listed include Hugging Face Hub, Replicate, the OpenAI API for chat and continuation models, AWS Bedrock, LiteLLM, anything reachable over REST, and gguf models via llama.cpp version 1046 or later. Scope is controlled with --spec. A family such as probes.promptinject selects the PromptInject framework's methods, and appending a plugin name after a dot selects one plugin, as in probes.lmrc.SlurUsage. garak --list_probes prints what is available.

Where the design gets uncomfortable

The weakest link is the detector, not the probe. A probe generates text; a detector decides whether that text counts as a failure. Every failure rate garak prints is therefore a claim produced by a classifier, a string match or some other judgement function, and the README does not describe how those detectors are validated. The project's own worked example shows how much this shapes the reading. The README presents results for the encoding module on a GPT-3 variant and on ChatGPT, and observes that the more recent model was much more susceptible to encoding-based injection, with text-babbage-001 found vulnerable only to quoted-printable and MIME encoding injections. That is a striking claim to hang on a detector, and it is exactly the kind of result a reader should want to reproduce before repeating. There is a second, more practical limitation. garak is a command-line tool developed on Linux and OSX, and it needs a live model to talk to. If your model is not reachable through one of the listed generators or a REST endpoint, or if policy forbids sending probe prompts to a hosted API, the tool has nothing to scan. A third constraint is scope drift. Running with no --spec tries every probe garak knows, which on a paid endpoint means an unbounded number of generations before you have asked a single question of your own. The narrow invocation is the sane one, and the README's examples are all narrow.

How it differs from a static prompt test suite

The obvious alternative is a hand-written suite of adversarial prompts checked into your repository, run in CI with assertions on the responses. The difference is in what each approach can represent. A static suite encodes the attacks you already thought of, and it stays exactly as broad as the last person who edited it. garak ships probe families drawn from published work, including the PromptInject framework and the Language Model Risk Cards behind probes.lmrc, and the README says it mixes static, dynamic and adaptive probes. That means the probe set can grow with the project rather than with your team's imagination, and versioned releases such as v0.17.0, v0.16.0 and v0.15.1 give you something to pin against. The cost of that breadth is interpretability. A static suite fails loudly on a named assertion you wrote and can explain. garak fails as a rate against a detector you did not write, and the failure rate moves when the detector changes, not only when the model does. If your goal is regression detection on a small set of behaviours you understand precisely, the static suite is the more honest instrument. garak earns its place when you want coverage of failure classes you have not enumerated yourself, and when you are willing to treat the output as leads rather than verdicts.

Maintenance, releases and the licence

garak is Apache-2.0, which is a permissive licence, and the repository is not archived. The last push recorded is 2026-09-09, the same timestamp as the v0.17.0 release, with v0.16.0 about a month earlier and v0.15.1 roughly three months before that. That cadence suggests active development rather than a frozen artifact, and it also means the probe and detector sets can shift between versions in ways that change your numbers. Anyone who reports a garak failure rate over time needs to record the version alongside it, and pinning is the only way to keep a comparison meaningful. The upgrade cost is mostly the cost of re-baselining: a new release may add probes, and running without --spec means new probes are included in the next run by default. The README does not document a migration or deprecation policy for probe and detector names, so treat a version bump as a reason to rerun your chosen spec and compare rather than assume continuity. On the licence itself, Apache-2.0 permits commercial use and modification, and I am not giving legal advice: read the licence text and your own obligations, particularly if you redistribute a modified garak or bundle it into a product.

Who should pick it up, and what to check first

garak fits a team that has a reachable model endpoint, a named failure class such as prompt injection or data leakage, and at least one engineer who will read the per-detector rows instead of the summary. It fits research and internal red-team work where reproducing a published probe is the point. It does not fit a compliance workflow that needs a binary attestation, because the tool reports rates against detectors and the detectors are not described as validated. It does not fit an air-gapped environment with no local model, since every run needs generations from somewhere. The first thing to verify is the detector behind whichever probe you care about, by reading what it counts as a failure and confirming that matches your policy. The second is the target: --target_type and --target_name must resolve to the exact model version you ship, and the README's own examples make clear that swapping a model name is a one-token change with a large effect on results. The third is the record. Errors land in garak.log and the run detail lands in the .jsonl file named at start and end, so decide where that file goes before the first scan, not after. Start with a single spec, for instance --spec probes.encoding, on a small local model, and only widen the scope once you can explain the rows you are looking at.

Editorial conclusion

Adopt garak if you have a model endpoint, a specific failure class in mind, and someone who can read a FAIL rate against a named detector rather than a single score. Do not adopt it if you need a pass or fail certificate for compliance, or if you cannot run the model at all and have no API budget, because the tool generates text and the README notes that more than one generation is made per prompt by default. Before trusting a run, verify three things: that the detector behind the probe actually measures what your policy cares about, that your target_type and target_name point at the same model version you ship, and that the .jsonl report is retained, since garak.log keeps errors and the jsonl keeps the run detail. Then compare two runs of the same spec on the same model before you draw any conclusion from a single failure rate.

Official sources

1. License: Apache-2.0
2. NVIDIA/garak on GitHub
3. Project website
4. README
5. Releases