Semia: static capability auditing for AI agent skills

The gap Semia targets: skills that run with your credentials

An agent skill is a markdown file that carries embedded shell commands, network calls, and tool invocations. The README states the execution context plainly: skills run with your credentials, on your machine, with your data. That combination is what makes a skill different from a library you install and inspect. The file you read is not the thing that acts. The host agent reads it and then acts.

The audience follows from that. Semia is for people who pull skills from a marketplace, a repository, or a colleague's pull request and need to decide whether to trust them. The README frames the choice as the difference between trusting a skill because the README looks fine and trusting it because Semia extracted 14 actions, 6 effects, and 2 secret reads, each grounded in a specific source line. That is the whole pitch: replace a reading of the prose with an inventory of capabilities.

It is not a runtime sandbox. Nothing in the material suggests Semia intercepts a skill while it executes. It is a pre-adoption review step, and that framing matters for where it fits in a workflow.

Four stages, one of which needs an LLM

The scan pipeline runs prepare, synthesize, detect, and report. Only synthesize needs a language model. The README says the other three stages are deterministic and require no key. That split is the most important architectural fact about Semia, because it tells you where variability enters.

Prepare normalizes the skill text and assigns stable line anchors, written to prepared_skill.md, with the reference units it aligns against in prepare_units.json. Synthesize converts the normalized skill into a behavior map expressed as Datalog facts, stored as synthesized_facts.dl. Detect evaluates rules over those facts and writes detection_findings.dl. Report renders the results.

The Datalog choice is worth pausing on. Findings are derived by rule evaluation over facts, not by asking a model to produce a verdict. That means the detection layer is inspectable: you can read the facts, read the rules, and see how one produced the other. The LLM's job is confined to producing the facts, which is the part no static parser can do reliably for natural-language instructions.

Repair extends the chain. It reads findings and synthesized facts from an existing scan, traces each violation back through the Datalog rules to identify the root cause, then calls an LLM to generate a SKILL.md patch. The patch either fixes the problematic content directly or adds specific security constraints. Output goes to patched/SKILL.md and repair_result.json.

Installing and running a scan

The CLI path is the shortest. Install with pip install semia-audit, then run semia scan ./some-skill. That single command does prepare, synthesize, detect, and report. Results land under .semia/runs/<skill-slug>/ by default, and --out <path> overrides the location.

Before that works you need a provider. The default is the OpenAI Responses API, so export OPENAI_API_KEY. The README notes the same path also works for DeepSeek, OpenRouter, and vLLM via OPENAI_BASE_URL. For Anthropic, set SEMIA_LLM_PROVIDER=anthropic and export ANTHROPIC_API_KEY, with ANTHROPIC_BASE_URL as the optional override. Two more options skip API keys entirely by shelling out to a local CLI login: SEMIA_LLM_PROVIDER=claude and SEMIA_LLM_PROVIDER=codex.

Model selection is a free-form string. Pass --model <name> on any scan invocation or persist it with SEMIA_LLM_MODEL. The README's examples include gpt-5.5, deepseek-v4, and claude-opus-4-7, and it states that anything the endpoint accepts is valid. Timeout and retry knobs, plus base-URL support and synthesis-loop tuning, live in ADVANCED_USAGE.md under Configuration.

Host plugins change the setup. Codex registers the marketplace with codex plugin marketplace add berabuddies/Semia, then needs [plugins."semia@semia"] with enabled = true in ~/.codex/config.toml. Claude Code uses claude plugin marketplace add berabuddies/Semia followed by claude plugin install semia@semia. OpenClaw is one command: openclaw plugins install clawhub:semia. In all three host cases the agent itself performs the synthesize step, so no API key is needed and the bundled semia.pyz handles prepare, detect, and report deterministically.

Reports, and why the SARIF path matters more than it looks

Every run produces report.md, with findings ranked by severity and each one tied to a source line. Two more formats are on demand: semia report .semia/runs/some-skill --format sarif emits SARIF 2.1.0, and --format json emits a structured payload containing check, evidence, and detector fields for programmatic consumers.

The SARIF output is the part with real workflow consequences. Because findings trace to specific lines, the README states the SARIF drops into GitHub Code Scanning and reviewers see annotations directly on the skill pull request. That turns Semia from a tool one person runs into a check that appears where the skill is being reviewed. The JSON format is the escape hatch for anything that is not GitHub.

The run directory also holds artifacts the README marks as internal: synthesized_facts.dl, detection_findings.dl, prepared_skill.md, prepare_units.json, synthesis_metadata.json, run_manifest.json, repair_result.json, and patched/SKILL.md. The README says most users only ever open the reports. That is probably right for routine review, but synthesis_metadata.json is the one internal file worth opening when a result looks wrong, since it records provider, model, retries, score, and stop reason for the synthesis step.

The synthesize step is the soft spot

Semia's accuracy depends on a language model turning prose into Datalog facts. Everything downstream is deterministic, so a missed capability in synthesis is not caught later by rule evaluation. The rules can only fire on facts that exist. If the model fails to record that a skill reads an environment variable, no detector will notice the omission, and the report will look clean.

The material does not describe a validation pass comparing synthesized facts against the prepared text, though prepare_units.json is described as holding the reference units the evidence text aligns against, which suggests some grounding mechanism exists. How strongly facts are constrained to those units is not stated. Treat this as the open question rather than a settled property.

The practical consequence is that a clean report is evidence of absence only as far as the synthesis was faithful. Before relying on Semia for a gate, scan a skill you already know is dangerous and confirm the detector flags it. That is a calibration step, not a formality.

There is a second limit. Semia reads skills as data and never executes them. A skill whose behavior depends on what it fetches at runtime, or on instructions assembled from a remote source, is outside what a static read can see. The README's own framing is about what a skill can do, and that phrasing is honest about the boundary.

Where Semia sits next to generic scanners

The obvious comparison is a general-purpose static analysis or secret-scanning tool pointed at the same directory. Those tools match patterns in files: a known token format, a suspicious import, a curl call. They do not need a model, they run offline, and they cost nothing per scan.

The difference is what counts as a finding. A pattern matcher flags a line because it resembles something bad. Semia builds a behavior map first, then derives findings by evaluating rules over that map, and attaches each finding to the capability it represents. That is why the README can talk about extracted actions and effects as distinct counts rather than a list of matched lines. The trade is that Semia needs an LLM and therefore needs either an API key or a host agent, while a pattern matcher does not.

For a directory of shell scripts, a pattern matcher is the better tool. For a natural-language skill that says to check the user's cloud configuration and then take an action, pattern matching has little to grip. Semia is aimed at the second case, and the Datalog layer is what makes its output reviewable rather than a model's opinion in prose. If you already run a secret scanner, the two are complementary: the scanner catches credential material, Semia catches intended behavior.

Repair mode and what it commits you to

semia repair .semia/runs/some-skill --from-scan reuses an existing scan. Running semia repair ./some-skill does scan and repair together. In both cases the tool traces violations back through the Datalog rules to a root cause, then asks an LLM to write a SKILL.md patch. The result is a proposed file at patched/SKILL.md plus repair_result.json describing the outcome.

Two things follow. The first is that repair inherits every weakness of synthesis: if the behavior map missed a capability, there is no violation to trace and no patch to generate. The second is that the patch is model-generated text destined for a file that an agent will read and act on. Reviewing it is not optional. A patch that adds security constraints changes the skill's instructions, and the README describes it as either fixing content directly or adding constraints, which are different kinds of edit with different review needs.

Repair is the feature most likely to be used casually and the one that most deserves a diff review before the patched SKILL.md is committed. The run directory keeps the original findings alongside it, so the trace from violation to patch is available if you want to check the reasoning.

Licence, maintenance, and what to verify first

Semia is Apache-2.0 and written in Python, requiring 3.11 or later per the pyproject badge. Apache-2.0 permits commercial use and modification and includes an explicit patent grant, with the usual notice and attribution conditions. That is a permissive licence, but it governs the tool, not the skills you audit or the patches repair generates. If you redistribute a patched SKILL.md, the licence of that skill is a separate question, and this is not legal advice.

The release cadence visible in the material is three tagged versions in the v0.1.x line between May and June 2026, with the repository last pushed in September 2026. A 0.1.x version number is a signal about API stability, and the README already points to ADVANCED_USAGE.md for configuration details, which suggests the surface is still moving. Upgrade cost is mostly the provider configuration: environment variables and the --model flag are the parts most likely to need attention, and SEMIA_LLM_MODEL persists a choice that a future default could change.

The verification order that follows from all this: confirm the tool runs against a skill with a known-bad capability, read synthesis_metadata.json to see which provider and model produced the facts, and only then wire the SARIF output into Code Scanning. The detector is the part you can trust to be consistent. The synthesize step is the part you are choosing to trust.

Editorial conclusion

Adopt Semia if you review third-party skills for Codex, Claude Code, or OpenClaw and you want findings tied to source lines rather than a summary. Skip it if you cannot send skill text to an LLM provider or if you need runtime enforcement, because Semia reads skills as data and never executes them. Before trusting a clean report, run semia scan on a skill you already know is dangerous and confirm the detector flags it, then inspect synthesis_metadata.json to see which provider and model produced the behavior map.

Official sources

1. berabuddies/Semia on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases