Niubi Guard: a dry-run-first abuse detection tool for GitHub maintainers

The problem Niubi Guard targets: coordinated Issue abuse, not ordinary spam

The README frames the project around a specific complaint from maintainers: hostile Issues, repeated copy-paste accusations, and reputation-pressure campaigns. That is a narrower target than generic spam filtering. A single junk comment is easy to ignore. A wave of near-identical accusations from several accounts is not, because each one looks plausible in isolation and the set only looks coordinated when you read them side by side.

The intended user is a GitHub maintainer who wants to keep the policy visible. The README states that you choose the detection signals, users, allowlists, model, prompts, confidence threshold, and response actions. Nothing is decided by a hidden classifier you cannot inspect. Every detection carries labels, matched keywords or usernames, AI confidence, reasons, evidence, and planned actions, so a false positive is traceable to the rule or the prompt that produced it.

The project also publishes threat material alongside the code: a fourth-wave GitHub Issue abuse report and an attack corpus under docs/. Whether that corpus reflects your repository's traffic is something you have to judge yourself; the README does not claim the patterns generalize to every community.

How the scan works: Octokit fetch, rule match, optional LLM classification, planned actions

The runtime is a TypeScript library with a CLI entry point and a Next.js console. The published package depends on @octokit/rest, so the scan path reads Issues and comments through the GitHub API rather than a webhook listener. The scan block in guard.config.json controls the scope: includeIssues, includeComments, state, since, and maxPages. That means a scan is a bounded, repeatable query, not a continuous daemon.

Detection happens in layers. Keyword rules from rules.keywords and rules.denyUsers match literally. rules.allowPhrases and rules.allowUsers carve out exceptions, which is where good-faith security disclosures and trusted maintainers are meant to land. An optional coldStartAccounts block enriches each actor profile and can flag interactions from accounts that look newly created, have an empty bio, and match a minimum number of signals; it is disabled by default.

The AI layer is separate and off by default. When llm.enabled is true, the adapter calls POST {baseUrl}/chat/completions and expects strict JSON back with malicious, confidence, label, reason, and evidence fields. The README notes that LLM detections default to review_only, and that auto_plan is the mode where high-confidence detections can generate planned actions from your enabled policy. The actions block is the last stage, and every entry in it starts as false.

Installing the niubi-guard CLI and running a first dry-run scan

The README gives an npm install path and a from-source path. The npm route is three commands: install globally, initialize a config, then scan against it. The scan command takes --config and points at the generated file.

npm install -g niubi-guard
niubi-guard init
niubi-guard scan --config guard.config.json

Running from a clone uses pnpm instead. The package.json declares pnpm@10.1.0 as the package manager, so the lockfile expects that toolchain. After pnpm install, the dev script runs the CLI through tsx, and a scan needs a GitHub token in the environment and a config file on disk.

git clone https://github.com/Albert-Weasker/niubi_guard.git
cd niubi_guard
pnpm install
export GITHUB_TOKEN=github_pat_xxx
pnpm dev -- init
pnpm scan -- --config guard.config.json

The web console is a separate process: pnpm dev:web, then open http://localhost:3000. The README notes that if that port is busy, Next.js will choose another port, so read the terminal output rather than assuming 3000. There is also a Docker path. The Dockerfile builds with pnpm, exposes 3000, and starts the web app with pnpm start:web, so the container serves the console, not the CLI.

docker build -t niubi-guard .
docker run --rm -p 3000:3000 niubi-guard

Before any of this produces useful output, guard.config.json has to name your repositories. The example file in the repository root is the starting shape, and rules.repositories takes owner/repo strings.

Configuring detection without enabling destructive actions

The config schema is the product surface. A minimal file sets repositories, a keyword list, and an allow list, and leaves llm.enabled false. This is the configuration to run first, because keyword matches are explainable without any model call.

{
  "repositories": ["owner/repo"],
  "rules": {
    "keywords": ["spam template", "copy-paste", "mass mention", "repeated link"],
    "denyUsers": ["suspicious-login"],
    "allowPhrases": ["good-faith report", "security disclosure"],
    "allowUsers": ["trusted-maintainer"]
  }
}

The actions block is where the risk lives. deleteComments, closeIssues, lockIssues, deleteIssues, blockUsers, and setInteractionLimits all default to false, and the README states that strong actions only happen when you configure them and run apply mode. Interaction limits have their own block with limit and expiry values, for example existing_users and one_month.

For the AI layer, the model field defaults to gpt-4o-mini, temperature to 0.1, and confidenceThreshold to 0.8. The systemPrompt in the example explicitly instructs the classifier not to flag good-faith criticism or valid reports. That instruction is doing real work: a semantic classifier told to find harassment will find it in blunt but legitimate bug reports unless the prompt draws the line. The README also states that API keys are not stored by the app and that the browser sends them only for the current scan request, which matters if you run the web UI on a shared machine.

What Niubi Guard does not do, and when to leave it alone

The scan is pull-based and bounded by maxPages, which defaults to 5 in the example config. On a repository with thousands of open Issues, that ceiling means older threads may never be examined in a single run. The README does not document an incremental cursor or a resume mechanism, so re-running with a since value is the apparent way to cover older ground.

The action set is also coarse. Closing an Issue, locking it, deleting a comment, and blocking a user are blunt responses to a problem that often involves one bad actor among several accounts. There is no documented appeal flow, no moderation queue with a second reviewer, and no rollback command for a block that turns out to be wrong. The README does not document rollback, so plan on undoing actions through the GitHub interface.

Finally, the cold-start heuristic is a proxy, not evidence. A new account with no bio and no avatar is a weak signal on its own, and the config requires minimumSignals precisely because one signal is not enough. Projects that attract first-time contributors, student cohorts, or throwaway accounts by design will generate noise here. If your repository's spam problem is a handful of messages a month, reading them yourself is faster than maintaining a config file, a token, and a model budget.

Alternatives: GitHub's own moderation controls and hand-rolled Actions

GitHub already ships interaction limits, user blocking, and repository-level permissions, and Niubi Guard's setInteractionLimits action ultimately calls that same mechanism with limit and expiry values. The difference is that GitHub's controls are manual and per-repository, while Niubi Guard reads the content first and proposes which threads and users deserve the limit. If you only ever need to slow down a known attacker, the native controls are fewer moving parts.

The other realistic alternative is a custom GitHub Action that greps Issue bodies for keywords and closes matches. That approach has no LLM cost, no config schema, and no dependency to update, and it is genuinely sufficient for template spam with a fixed signature. What it cannot do is explain why a match fired across a set of accounts, or classify an accusation that contains no fixed keyword. Niubi Guard's value sits in that middle ground: the matched keywords and usernames, the AI confidence, and the allow lists are all recorded per detection, which is the part a twenty-line shell script does not give you.

Maintenance cost, licence status and upgrade surface

The repository was last pushed on 2026-08-27, and the only release listed is v0.1.0 from 2026-06-04. That is a young project with a single tagged version, so treat the config schema as something that can move. The package.json declares zod for validation and @octokit/rest for the GitHub client, both of which are active dependencies you inherit.

On the licence: the repository description lists the licence as NOASSERTION, while the README links to ./LICENSE and states Apache-2.0, and the LICENSE file is present at the repository root. Those two signals disagree, and NOASSERTION is what GitHub reports when it cannot match the file to a known licence. Read the LICENSE file directly before you rely on either label. This is not legal advice, and if you plan to redistribute the code or ship it inside a commercial product, the file itself is the thing to check.

Upgrade cost is mostly schema drift. The published package ships guard.config.example.json, README.md, README.zh-CN.md, CHANGELOG.md, and LICENSE alongside dist, so a version bump can change both the CLI behaviour and the example config in the same step. Diffing the example file against your own guard.config.json after each upgrade is the concrete check, since unknown or renamed keys are validated by zod and may fail the scan outright.

Editorial conclusion

Niubi Guard fits maintainers of public repositories who are already fielding coordinated Issues and want the detection logic visible in a config file before anything is deleted. It is the wrong tool for a maintainer who wants an automatic ban hammer on day one, and it is overkill for a repository with no spam problem. Before enabling actions.deleteComments or actions.blockUsers, run the scan in dry-run, read the labels and confidence values, and confirm that rules.allowPhrases covers your project's normal security and bug-report vocabulary.

Frequently asked questions

Official sources

1. Albert-Weasker/niubi_guard on GitHub
2. Issues
3. Project website
4. README
5. Releases