CLI tool
Tommy-yw/RunbookHermes avatar
Tommy-yw/RunbookHermes

RunbookHermes: an evidence-first AIOps layer bolted onto the Hermes agent runtime

Hermes-native AIOps agent for evidence-driven incident response, approval-gated remediation, and runbook learning.

513 stars37 forksPythonMIT

At a glance

What is it?
RunbookHermes keeps the Hermes Agent runtime and adds an incident domain layer on top: evidence collection before root cause, a fixed approval chain for dangerous actions, and a memory, RAG, skill and training loop that runs after every incident. It is a Python and Node repository with a Web Console, five intake entry points and no published releases.
Who is it for?
RunbookHermes is worth reading if your incident tooling currently asks a model to guess a root cause and then trusts it, because the ordering it enforces is the part worth copying: evidence first, memory as a weak prior, destructive actions behind approval and a checkpoint, and a written residue after every incident.
Can I use it commercially?
Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Activity is slowing. The repository last received commits 6 months ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • Tommy-yw/RunbookHermes513 ★ · -32
Star history
Repository2026-09-212026-11-15
Tommy-yw/RunbookHermes545513

-32 Stars in the last 55 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

RunbookHermes is a domain layer bolted onto a vendored agent runtime

RunbookHermes is not a standalone agent framework. It keeps the Hermes Agent runtime and stacks an AIOps domain layer on top, which is the composition the project states as Hermes Agent Runtime plus RunbookHermes AIOps Domain Layer equals RunbookHermes. What survives from the upstream side is the agent main loop, the tool registry, multi-model providers, CLI and Gateway entry points, the MemoryProvider, Skills, context compression, session state and RL trajectory collection. What gets added is the incident half: intake, Prometheus, Loki, trace and deploy evidence, an EvidenceStack context, an RCA guard, an action policy, approval with checkpoint and dry-run, controlled execution, recovery verification, self-evolving memory, citation RAG, multimodal evidence, an eval benchmark, a training and RL dataset pipeline, a Web Console, and Feishu, WeCom and Alertmanager entry points. The top level of the repository reads like an upstream checkout plus that layer, with hermes/, hermes_cli/, hermes_constants.py, hermes_state.py, hermes_time.py, hermes_logging.py, gateway/, acp_adapter/, acp_registry/, plugins/, profiles/, optional-skills/, cron/, integrations/, environments/, nix/, apps/, demo/, batch_runner.py, cli.py, rl_cli.py, mini_swe_runner.py and mcp_serve.py. HERMES_BASELINE.lock sits at the top with them, and a demo/payment_system/ tree exists as a worked example.

Evidence first: eight sources collected before any root cause is written

The ordering is the product. RunbookHermes does not write a root cause until it has collected metrics, logs, traces, deploy history, service profile, memory context, RAG citation and multimodal evidence, and only then generates the RCA. Deploy history sits in that list next to the live signals, which is the difference between an agent that notices a latency regression and one that knows a deploy preceded it. The console makes the same ordering visible. The realtime monitoring page covers payment-service, coupon-service and order-service with HTTP status signals, QPS, p95 latency, service topology, backend mode and deployment state, and the lower half of that page carries the log and trace signals. The incident detail page turns those into evidence cards from metrics, logs and traces, then attaches an executive summary naming the root cause, the recommended action, the evidence IDs, a confidence figure and the approval status. On the root cause tab the deterministic evidence is kept apart from the optional model-assisted explanation, so a reviewer can see which half of the write-up came from a signal and which half came from a model.

Memory is a weak prior that never becomes the conclusion

Memory has a defined rank here and it is not first. Historical memory can remind the agent that a service has failed this way before, and the project states that the conclusion still has to come from the metrics, logs, traces and deploy evidence of the incident in front of you. That is a guardrail against a system that pattern matches its way to a confident wrong answer. The memory console exposes the local memory layer and names the mechanisms behind it: Prompt Notebooks, SQLite FTS5 memory search, HRR-based offline semantic recall, skill indexing, trust scores and memory feedback. Operators write into that layer directly, adding service profiles, rollback rules, approval preferences and recurring failure patterns that are stable operational facts rather than chat transcripts. FTS5 handles the exact-match half of retrieval, the HRR pass handles a semantic recall that runs offline, and trust scores sit above both. The consequence is that memory widens what the agent can search while the evidence requirement keeps it from deciding anything on its own.

Seven action classes sit behind approval, checkpoint and dry-run

Dangerous actions are named as a list rather than as a category. Rollback, restart, scaling mutation, traffic switching, config mutation, database-affecting operations, cache flush and dependency failover are the classes that cannot be executed straight from a model response, and each one goes through the same chain:

text
action policy
→ approval
→ checkpoint
→ dry-run
→ controlled execution
→ recovery verification
→ audit timeline

Read the chain as an ordering constraint rather than as a menu. A checkpoint is written before the change, so the timeline has something to point at if the change needs undoing. The dry-run step evaluates the action before it touches the system. Recovery verification is its own stage with its own pass condition, and the audit timeline closes the sequence. The approval center is where a person stands in the chain: it shows the action, the risk level, the checkpoint and the payload, and the operator approves or rejects from there. The incident timeline records the same stages in order, from incident creation and evidence collection through hypothesis generation, action planning, checkpoint creation, approval request, approval decision, skill generation and execution result, so a rejected action leaves as much of a trace as an executed one.

Each incident deposits a skill, an eval case and a reward label

Once an incident closes, the residue is written down in nine forms: incident summary, fault pattern, service governance rule, team runbook habit, a generated SKILL.md, a RAG document, an eval case, a training trajectory and a reward label. The generated SKILL.md is the piece with operational consequence, because it turns one resolved failure into a reusable skill instead of a chat log that disappears. The eval case and the reward label close a loop that the upstream runtime only half had. RL trajectory collection exists in the Hermes runtime, and rl_cli.py, datagen-config-examples/ and batch_runner.py at the top level are what turn incidents into training data. RAG is the other half of the memory story, and it is citation-based: SRE manuals, service troubleshooting documents, architecture notes and runbook content are ingested into a local knowledge base whose search results carry source and chunk citations, so an operational recommendation points back at where it came from instead of arriving unsupported. One gap is worth naming. The console section describing the Benchmark and Eval page stops mid-sentence, so the measurement that would tell you whether any of this is getting better is the one part of the write-up it does not finish.

The model summary is optional, the evidence stack is not

A model provider is optional in a specific, documented way. The root cause tab separates deterministic evidence from the model-assisted summary, and that summary is only enabled when a provider is configured, so an incident can be trialled with no model at all and still produce an evidence record. The provider setup in the environment template covers three routes. OpenRouter is the default path, with the default model living in config.yaml under model.default and changed with the hermes model or hermes setup command rather than through an environment variable, which the template notes explicitly. Google AI Studio is reached through Google's OpenAI compatible endpoint using GOOGLE_API_KEY, with GEMINI_API_KEY as an alias. Ollama Cloud is the third route, pointed at its own OpenAI compatible base URL. The settings page then reports, per integration class, whether the model, observability, execution, Feishu and WeCom interfaces are actually wired up, and lists the environment variables each one needs. That readiness view is where you find out how much of the system is live on a given day.

Five entry points, one normalized incident list

Incidents arrive from five directions: the Web console, Alertmanager, Feishu, WeCom and the API. The incident list page normalizes all of them into a single table, so a Feishu alert and a hand-created ticket show up with the same fields, service, status, severity, root cause and creation time, and the same quick-create action. The overview page is the shift-level view: incident count, pending approvals, generated skills, critical services, a recommended operation flow, the current capability boundaries and a live monitoring preview. Listing capability boundaries next to the incident count is a deliberate choice, since an operator looking at a pending approval needs to know what the system will not do on its own. The digests and skills page closes the response side by summarizing recent incidents and high-frequency faults next to the runbook skills generated from them, which is the same material the memory console exposes for search, arranged for review instead of retrieval.

Supply chain pinned in the manifest, and two disagreeing versions

The packaging is where the care in this project is easiest to see. The Python manifest pins its core dependencies to known-good ranges with the stated reason of limiting the supply chain attack surface, and two of those pins carry an inline CVE reference: requests at 2.33.0 or newer and below 3 for CVE-2026-25645, and PyJWT with crypto extras at 2.12.0 or newer and below 3 for CVE-2026-32597. Requires Python is 3.11 or newer, with optional extras for Modal and Daytona and groups for dev, messaging, cron, slack and matrix. The Node manifest is minimal by comparison, two browser tool dependencies and an engines floor of Node 20, and it names the package hermes-agent at version 1.0.0 while the Python manifest names hermes-agent at 0.11.0. Two versions of the same package name in one repository is a small but real wrinkle when you are pinning an install. The Dockerfile pins its toolchain by digest, uv 0.11.6 on Python 3.13 trixie and gosu 1.19 on trixie, over debian 13.4, runs as a non-root user with UID 10000 and installs browsers through playwright limited to chromium in shell mode. There are no GitHub releases, and the last push is dated 2026-05-18.

Editorial conclusion

RunbookHermes is worth reading if your incident tooling currently asks a model to guess a root cause and then trusts it, because the ordering it enforces is the part worth copying: evidence first, memory as a weak prior, destructive actions behind approval and a checkpoint, and a written residue after every incident. It is a weaker fit if you want a tool you can install and run this afternoon, since there is no tagged release, the install surface spans Python 3.11, Node 20 and a digest-pinned Debian image, and the write-up never finishes describing how the system is measured. Before depending on it, check three things on your own cluster: which of the five intake paths you can actually wire up, whether your observability stack matches the Prometheus, Loki, trace and deploy sources the evidence layer expects, and who is allowed to sit in the approval loop at 3am.

Frequently asked questions

What is RunbookHermes actually built on?

It keeps the Hermes Agent runtime, which supplies the main loop, tool registry, multi-model providers, CLI and Gateway entry, MemoryProvider, Skills, context compression and RL trajectory collection, and adds an AIOps domain layer for incident intake, evidence, RCA, gated actions and the runbook memory loop. Both manifests still declare the package name hermes-agent, and HERMES_BASELINE.lock sits at the top of the repository.

Does RunbookHermes need a model provider to work?

No. The root cause tab separates deterministic evidence from the model-assisted summary, and the summary is only enabled when a provider is configured. The environment template covers OpenRouter, Google AI Studio and Ollama Cloud as the three provider routes.

How does RunbookHermes decide an action is safe to run?

Seven action classes, including rollback, restart, scaling mutation, traffic switching, config mutation, database-affecting operations, cache flush and dependency failover, go through the same fixed chain: action policy, approval, checkpoint, dry-run, controlled execution, recovery verification and audit timeline. A person approves or rejects in the approval center after seeing the action, risk level, checkpoint and payload.

What does RunbookHermes leave behind after an incident is closed?

Nine artefacts: an incident summary, a fault pattern, a service governance rule, a team runbook habit, a generated SKILL.md, a RAG document, an eval case, a training trajectory and a reward label. The citation RAG base and the skill index keep the operational half of that searchable.

Is there a RunbookHermes release to install from?

The repository publishes no GitHub releases, and its last push is dated 2026-05-18. What is pinned instead is the toolchain: the Dockerfile pins uv 0.11.6 and gosu 1.19 by digest over debian 13.4, the Python manifest requires 3.11 or newer, and the Node manifest requires Node 20 or newer.

Official sources

  1. Issues
  2. License: MIT
  3. README
  4. Tommy-yw/RunbookHermes on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/tommy-yw-runbookhermes.svg)](https://hysenlabs.com/projects/tommy-yw-runbookhermes)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/tommy-yw-runbookhermes.svg)](https://hysenlabs.com/projects/tommy-yw-runbookhermes)

Was this useful?