T3MP3ST: a multi-agent red teaming harness that runs on the coding agent you already have

The problem T3MP3ST targets: red teaming is gated by tooling cost

Offensive security has a tooling problem before it has a talent problem. A working web-app engagement means a proxy, a fuzzer, a payload library, a notes system and a report generator, each with its own configuration, and someone who knows how to wire them together. The README states the bet directly: offensive security sits behind years of practice and expensive tooling, and a coordinated agent swarm could put real bug-hunting in reach of people who never had that background.

The audience follows from that. This is not a tool for a large security team that already owns a mature stack. It is for a developer or small team that already runs an AI coding agent and wants to point it at an authorized target without buying a second subscription. The README frames the agent as the brain and T3MP3ST as the war machine bolted around it, which is a fair summary of the architecture: the project supplies orchestration, not the model.

That framing also sets the ceiling. If you do not already have a coding agent installed, or a key for one of the supported providers, T3MP3ST has nothing to drive.

How the agent swarm is wired: kill chain, War Room and the model fallback

The pipeline is recon, exploit, report, run either from a CLI or from a browser interface the README calls the War Room. You describe a target to a component named Op Admiral in plain English and launch; the connected agent executes the stages.

The repository layout backs this up. src/ holds the TypeScript sources, scripts/ holds the operator-facing Node scripts (doctor.mjs, verify-claims.mjs, model-matrix.mjs, exploit-chain-smoke.mjs, arsenal-doctor.mjs), and migrations/ and tenants.json suggest a persistent store with some notion of separate tenants. A Dockerfile and docker-compose.yml exist for a containerized deployment, and the Dockerfile sets DOCKER=true so the server binds to 0.0.0.0 instead of loopback.

The supported backbones are the interesting design choice. Claude Code, Codex, Hermes, OpenCode and Oh My Pi are named as local agents; Ollama, LM Studio and vLLM are named for fully offline runs; OpenRouter, Venice, Anthropic, OpenAI, DeepSeek, Hugging Face, Novita, Groq, Together, Replicate and xAI appear in .env.example. Because the agent you are already signed into does the reasoning, no new API key is strictly required. The README also advertises a model fallback path, with a test script at scripts/test-model-fallback.mjs, though the README does not document the fallback ordering or what triggers a switch.

Target authentication is handled through two environment variables, TEMPEST_TARGET_ORIGIN and TEMPEST_TARGET_HEADERS, both required together. The origin must be an exact scheme plus host plus optional port, and headers are a JSON object injected only into requests to that origin, with per-request headers overriding the defaults case-insensitively. That is a narrower design than a global header file, and deliberately so.

Installing T3MP3ST and launching a first mission

The README's quick start is two commands and a browser tab. It states the setup takes about two minutes, and that mission time depends on the target.

npm install
npm run server        # War Room → http://127.0.0.1:3333/ui/

After npm run server, open http://127.0.0.1:3333/ui/ in a browser. The README then says to open Settings and connect a local agent (Claude Code, Codex, Hermes, OpenCode or Oh My Pi), describe a target to Op Admiral in plain English, and launch. No key is required for that path.

If you would rather bring a provider key, the README gives this alternative and says you can skip the connect step:

export OPENROUTER_API_KEY=...     # or VENICE_API_KEY / ANTHROPIC_API_KEY / OPENAI_API_KEY
export XAI_API_KEY=...            # Grok Build (grok-build-0.1) — xAI's coding model, native tool-calling
export NOVITA_API_KEY=...         # Novita AI's hosted OpenAI-compatible API

The same variables are laid out in .env.example, which instructs you to copy it to .env and never commit the real file. Note the README's warning about Novita specifically: requests send the selected model id, prompts, context, generated output and request metadata to that provider, and the README states T3MP3ST does not claim independent assurance for sensitive workloads. For a tool that handles target data, that sentence is worth reading twice before you paste a client's hostname into a prompt.

For a containerized run, docker-compose.yml builds the app service, mounts ./reports and ./evidence, and publishes 127.0.0.1:3333:3333. The healthcheck polls http://localhost:3333/api/health every 30 seconds with a 40 second start period. The Dockerfile is based on node:22-alpine pinned by digest and runs npm ci, npm run build, then npm run server.

One environment variable appears in the truncated README as a way to give slow local agents more room: T3MP3ST_LOCAL_AGENT_T, cut off mid-name. Do not guess the full key; check .env.example or the source before setting it.

What the status table admits: cloud, mobile and binary are scaffolding

The most useful part of the README is the table that separates live from planned, and it is unusually candid. Web apps, CTF and the robotics/OT/embedded disclosure pipeline are marked stable. Source-code analysis is marked stable with multi-language ingest through web-tree-sitter. Smart contracts are marked as reproduction rather than novel discovery, which is a real downgrade: running Damn Vulnerable DeFi challenges proves the harness can follow a known path, not that it finds anything new.

Then there are three rows marked with a construction marker. Cloud IaC has a misconfig benchmark called cloud:bench and an opt-in arsenal (aws, az, gcloud, scoutsuite, cloudfox, pmapper, with pacu gated), but the README states live-cloud exploitation is not yet benchmarked. Mobile has a static analyzer for manifest misconfiguration and secret or cleartext detection plus mobsfscan, objection and drozer, with frida gated, and dynamic exploitation is not benchmarked. Binary and reverse engineering has a decompiled-output sink detector for unsafe copy, format string, command injection and integer overflow, plus ghidra, radare2, objdump, checksec and strings, with gdb gated, and solving or pwn is not benchmarked.

Read that as a boundary, not a footnote. If your engagement is cloud, mobile or binary, the README is telling you the detection scaffolding exists and the exploitation claims do not. A team that skips the table and reads only the headline benchmark will be disappointed on exactly those three domains.

The gating on pacu, frida and gdb is also worth noting. Those are the tools that turn static detection into live action, and the README describes them as gated rather than available, without documenting the gate.

The reproducibility claim and where the README goes quiet

T3MP3ST's central marketing claim is that its numbers recompute. The README states that every figure re-derives from committed data with npm run verify-claims, that the badge reads 27/27, and that a claim which cannot be reproduced does not ship. The headline figure is 90.1% pass@1 on XBOW's 104-challenge suite, placed above XBOW's self-reported 85%, alongside hint-free CTF solves and a cold hunt on post-cutoff CVEs.

That is a testable claim, which is more than most projects offer, and the script exists in package.json alongside bench:model-matrix, cve:bench, cloud:bench and a set of smoke tests. But the README does not explain the methodology behind the 90.1% figure: how many attempts, which model, which agent backbone, or whether the number holds across the five supported local agents. The verify-claims script proves the arithmetic re-derives from committed data; it does not prove the committed data came from a fair run. Those are different guarantees, and the README blurs them.

Other gaps are plainer. There is no documented rollback or uninstall path. The README does not state what happens to reports and evidence after a run beyond the two mounted directories in docker-compose.yml. The truncated environment section means at least one documented variable is unreadable in the excerpt. And the repository ships a WHITEPAPER.md and VISION.md, which suggests the deeper design rationale lives outside the README rather than in it.

There is also a hidden flag mentioned in an HTML comment in the README, tied to npm run verify-claims. It is flavor, not a feature, but it tells you the project expects people to actually run the verification script.

T3MP3ST versus a conventional pentest suite

The obvious alternative is a conventional offensive toolchain: Burp Suite or ZAP for web interception, Metasploit for exploitation, nmap for recon, and a human writing the report. The difference in approach is where the intelligence sits. In that stack, the human decides what to try next and the tools execute; in T3MP3ST, a language model decides what to try next and the harness executes, with the human describing the target and reading the output.

That changes failure modes. A conventional suite is deterministic: the same scan against the same target gives the same result, and a miss is a miss you can reason about. An agent-driven harness is probabilistic. The README's own benchmark framing, pass@1, concedes this: the metric is about whether the first attempt succeeds, which is only a meaningful number if attempts vary. Two runs against the same target can diverge, and the README does not document how a partially successful chain is resumed or replayed.

It also changes cost structure. A conventional suite is a licence you buy once. T3MP3ST is keyless if you route through a local agent you already pay for, which is the pitch, but the moment you set OPENROUTER_API_KEY or similar you are back to metered inference, and an agent that iterates on payloads burns tokens quickly. The README advertises the keyless path without pricing the keyed one.

Where the conventional stack still wins: anything requiring a stable, auditable record of exactly which requests were sent. T3MP3ST writes reports and evidence to mounted directories, but the README does not describe a request-level audit log.

Licence, upgrade cost and what AGPL-3.0 means here

T3MP3ST is licensed AGPL-3.0. The README states it is provided as-is with no warranty and no liability for damage, loss or misuse, and that users are responsible for staying inside the law and their rules of engagement. The authorized-use section is explicit that the tool is for systems you own or have written permission to test.

The AGPL detail that matters for a security team is the network clause. If you modify T3MP3ST and expose it to users over a network, the AGPL's source-availability obligation can attach to your modified version. A browser War Room served on 127.0.0.1:3333 for your own use is not the same situation as an internal service other people reach. This is not legal advice; if you plan to fork it into an internal platform, have counsel read the licence rather than this paragraph.

Upgrade cost is currently low. The repository's most recent release is v1.0.0, described as a certified source checkpoint and dated 2026-09-08, and the last push to main was on 2026-09-08. There is no long release history to reason about yet, so there is no established pattern for breaking changes. The presence of migrations/ and tenants.json means an upgrade may involve a schema migration, but the README does not document an upgrade procedure. .env.example is a good upgrade reference: compare it against your .env after pulling, since new provider keys appear there.

Editorial conclusion

Adopt T3MP3ST if you already run a local coding agent, you have written authorization for the target, and you want the recon to exploit to report loop driven from a browser War Room at 127.0.0.1:3333/ui/ rather than assembled by hand. Do not adopt it for cloud, mobile or binary engagements: the README's own status table marks those as scaffolding with no benchmarked live exploitation. Before committing, run npm run verify-claims to confirm the 27/27 badge re-derives on your machine, and check the status table for whichever domain you actually care about.

Frequently asked questions

Official sources

1. elder-plinius/T3MP3ST on GitHub
2. Issues
3. License: AGPL-3.0
4. README
5. Releases