nlpm: a linter that scores the markdown files driving coding agents

Scores start at 100 and fall by fixed penalties

The scoring model is the easiest part of this project to understand and the easiest to argue with. A score begins at 100 and every issue found subtracts a fixed amount, so the same artifact with the same problems always produces the same number, which the readme states as a property of the design rather than a side effect. The bands are then simple: 90 to 100 is excellent and production-ready, 80 to 89 is good with minor gaps, 70 to 79 is adequate and should improve, 60 to 69 is weak and below threshold, and anything under 60 is a rewrite. The default pass threshold is 70, and it is changed in a local markdown file in the project, so the number a project enforces is a decision rather than a constant. The full penalty tables live in a scoring file, with separate tables for other tools, memory files and workflow programs indexed from it.

The check it claims nobody else runs: manifest against disk

One check is the reason this project exists, and the readme is explicit that it is the differentiator. The bug class is a skill file that exists on disk but is silently absent from the plugin manifest, which means it is invisible after a user installs the plugin. Nothing errors; the artifact is simply never loaded. The readme says this was verified across more than eight tools, naming the official validator and one from the Linux Foundation among them, and it claims no other validator covers it. That claim is documented in a research file in the tree rather than asserted in a footnote, which is the right place to look if you want to check it. The broader framing is the analogy the project leans on: a linter for JavaScript, a linter for Python, and now one for the markdown that drives agent behaviour.

Eight slash commands, and one file for everyone else

Inside a coding agent the tool is eight slash commands, and each does one thing: list and inventory the artifacts in a repository, score them, run cross-component consistency checks, auto-fix what is fixable, track score trends over time, run artifact tests against spec files, initialise the tool for a project, and scan plugins for security risks in executable artifacts. Scoring accepts a path or a changed-files flag. Everything above requires the agent runtime, which is why there is a second path. The standalone validator is a single Python file with no external dependencies, and the readme stresses that it runs the deterministic subset of the consistency check, exiting non-zero on high-confidence findings. It installs with a two-line fetch and a chmod:

curl -fsSL -o /usr/local/bin/nlpm-check \
  https://raw.githubusercontent.com/xiaolai/nlpm/main/bin/nlpm-check
chmod +x /usr/local/bin/nlpm-check
nlpm-check .               # exit 1 on high-confidence findings

Drop-in templates for a pre-commit hook and an Actions workflow are also provided.

Two marketplaces, and the maintainer's own can be stale on disk

There are two install routes and they differ in a way that catches people out. The first adds the vendor's curated community marketplace, and the readme warns that updates there lag the maintainer's own marketplace by up to about a day. The second adds the maintainer's marketplace, where the newest version lands first, and installs with either project or user scope, with project scope marked as the recommendation. The failure mode is documented as a specific error message about a plugin not being found, and the cause is a stale local clone of the marketplace rather than a missing plugin. The fix is to update the marketplace explicitly and retry, because the install command does not refresh the clone on its own. The readme notes that the community marketplace does not have this caveat, which is a small argument for the slower path if you hit it.

The rubric is per-tool, across three ecosystems

The word multi-tool in this project's claims refers to something specific: the scoring rubric covers three ecosystems through what the readme calls tier-aware overlays, those being the Claude Code plugin format, the Codex command line format, and a third one. The same structural fault is expressed differently in each, so the penalty tables are not shared, and the readme says outright that the tables for the second and third ecosystems, for memory files and for workflow programs, are kept in a references directory and indexed from the main scoring file. This is also why the manifest-versus-disk check matters beyond one runtime: a manifest format that differs per tool is exactly the kind of thing that gets updated in one place and forgotten in another. The rubric's per-tool structure is the answer to that, at the cost of needing to read the right table.

It audits other people's plugin repositories and writes back

Past the linting, the project describes itself as a self-evolving pipeline running in continuous integration, and the mechanism is worth understanding because it explains why the rule set grows. The pipeline audits real plugin repositories, contributes fix pull requests to them, harvests teaching examples from the ones that come out clean, and feeds what it learns back into its own rule catalogue. The exemplar stage does the harvesting: any repository auditing clean at a score of 90 or above produces a teaching artifact, and the readme counts 62 published so far, covering 31 of the 50 rules with real-world positive references. The citation stage then runs weekly and opens a pull request that adds a real-world example link next to each rule, with a human approving the merge. So each rule ends up documenting a bad case in its own body and a good case in a repository somebody actually runs.

The pipeline found 990 mislabelled rule IDs in its own history

The most revealing number in the readme is a self-inflicted defect. A two-stage detector re-validates every audit's rule identifier twice: once against the rubric, which catches a value of the wrong type, and once against the rule's own title keywords, which catches an identifier that is the right shape but points at the wrong rule. A sweep on May 13, 2026 found 990 mislabelled identifiers across 128 historical audits. That is a substantial fraction of the audit history being scored against the wrong rule, and the readme says the detector is now wired in as a soft warning in every new audit so the drift is caught as it happens rather than found later. A companion script then calibrates the per-rule health view, reporting validated hits as raw hits minus drift hits alongside the number of exemplars for that rule, so the rules that look broken are measured against real violations rather than labelling noise.

A plugin repository that is also a documentation website

The tree explains the rest. Alongside the readme are per-tool plugin manifests and per-tool configuration directories for three ecosystems, which is how one repository ships to all of them, plus rule, example, privacy and agent instruction files at the root and a badge file. The working directories split by role: the skills the tool itself uses, the auditor with its scripts and exemplars, the standalone binary, case studies, hook and command definitions, the documentation, the template directory, the tests, a self-test directory, and a site directory. That last one is why the repository's reported primary language is HTML rather than Python, since the tool itself is one Python file plus a small tree. The licence is ISC, there are no tagged releases, and the last push to the main branch is dated September 30, 2026.

Editorial conclusion

Use nlpm if you publish plugins or skills for a coding agent and want a deterministic score plus a check for artifacts that exist on disk but not in your manifest, since that is the failure it was built around. Do not use it as a general writing grader, because every penalty is a fixed number attached to a specific structural fault rather than a judgement about prose, and do not expect the model-driven commands to run in CI, since the CI path is the separate standalone file. Before you wire it into a pre-publish gate, read the penalty tables for your own tool, since the rubric is per-tool, and set the pass threshold deliberately rather than inheriting the default.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository