aur-sleuth: An LLM Audit Gate for AUR Packages Before makepkg Runs

The specific gap it fills in the AUR workflow

The AUR is a community repository where anyone can upload a PKGBUILD. A malicious maintainer can hide code in the packaging scripts or in a downloaded source file. aur-sleuth targets exactly that attack: it answers whether it is safe to run makepkg on a given package and install the result. It does not evaluate the upstream application's quality or data-handling practices. That boundary is explicit in the README, which states it is not a general security auditor and not a vulnerability scanner. The intended user is an Arch user who uses yay or makepkg regularly and wants an automated review step before executing build scripts.

How the audit works: agentic LLM inspection of source files

The tool fetches the PKGBUILD from the AUR, then retrieves all files listed in the source array. It also pulls any other files from the package sources that the auditing LLM deems interesting. The README calls this an agentic security audit, meaning the LLM decides which additional files to examine. Each file gets a per-file status, and an overall result is derived. The statuses are SAFE, UNSAFE, INCONCLUSIVE, and SKIPPED. SKIPPED is for files detected as binary, which reduces audit coverage but does not fail the audit by itself. INCONCLUSIVE covers LLM or API errors, malformed model output, and similar failures. By default, both UNSAFE and INCONCLUSIVE cause a non-zero exit code, so the build is blocked unless you set AUDIT_FAILURE_FATAL=false.

Getting it running: install, config, and the yay hook

Installation is straightforward. You can install from the AUR itself with yay -S aur-sleuth-git, or manually by installing the uv dependency, cloning the repo, and running sudo make install. A user-local install is possible with make install PREFIX=$HOME/.local, which puts the scripts in $HOME/.local/bin. Configuration is done through environment variables or an INI file at ~/.config/aur-sleuth.conf. The required variable is OPENAI_API_KEY. OPENAI_BASE_URL defaults to OpenRouter if not set, and OPENAI_MODEL defaults to qwen/qwen3-235b-a22b-2507. You can adjust concurrency with MAX_LLM_JOBS, temperature with LLM_TEMPERATURE, top-p with LLM_TOP_P, and reasoning effort with LLM_REASONING_EFFORT. The README gives examples for OpenRouter, OpenAI, and a local ollama instance, the latter using OPENAI_API_KEY=ollama and a localhost URL.

Integration with makepkg and yay: the wrapper approach

The key integration is a shell wrapper named makepkg-with-sleuthing. It runs the audit first, then, if the audit passes, proceeds with the actual build and install. You can invoke it directly with makepkg-with-sleuthing -si in a directory containing a PKGBUILD, or through yay with yay --makepkg makepkg-with-sleuthing package-name. To make it the default for all yay operations, the README suggests yay --makepkg makepkg-with-sleuthing --, after which plain yay commands will audit installed or updated packages automatically. This is a clean design because it does not modify makepkg itself. It simply replaces the makepkg command that yay calls, which is a documented yay feature. The wrapper is the only place where the audit result becomes a build gate.

A real limitation: false positives and the fatal-by-default design

The README is honest about a major limitation: the audit is highly model dependent, and false positives can occur. It even suggests setting AUDIT_FAILURE_FATAL=false if you want to see the audit results without blocking the install, specifically because of a high rate of false positives. That means out of the box, an INCONCLUSIVE result, which can happen from a transient API error, will halt your build. This is a sharp trade-off: safety against availability. For a user who installs many packages, a single malformed model output could stop an update until you intervene. The tool does not appear to have a retry mechanism or a quarantine mode. You either trust the LLM verdict or you disable the fatal behavior and lose the protection.

The alternative: manual review or a non-LLM linter

The obvious alternative is the traditional practice of reading the PKGBUILD yourself before running makepkg. That is what the Arch Wiki has always recommended, and it remains the baseline. Another alternative is a static analysis tool that uses pattern matching rather than an LLM, such as a shellcheck wrapper or a custom grep-based audit script. The difference in approach is fundamental: aur-sleuth uses a general-purpose language model to interpret code, which can catch novel or obfuscated patterns but can also hallucinate or miss things. A pattern-based tool is deterministic and fast, but it only catches what its rules know. The README does not compare aur-sleuth to any specific tool, so you are left to weigh the probabilistic nature of the LLM against the deterministic but limited nature of rule-based checks.

Maintenance, licensing, and maturity concerns

The repository has no tagged releases, and the license field is unknown. That is a red flag for a security tool you might run before installing software. The default branch is master, and the project is not archived, but the last push date is not given in the material. The README references a static dashboard at mgalgs.io/aur-sleuth generated from an audit-reports branch, which suggests the author runs it regularly. Still, the lack of a license means you cannot legally redistribute or modify it without asking, and the absence of releases means you are tracking the master branch. For a tool that gates your package installs, you should verify the license and check the commit history before adopting it. The maintenance cost is low if you use the AUR package, but you must monitor API usage and keep your API key secure, as the README warns.

Editorial conclusion

Adopt aur-sleuth if you install many AUR packages and want an automated first-pass review that catches obvious malicious code before makepkg runs. It is not for you if you need a comprehensive vulnerability scanner or if you distrust LLM output enough to require zero false positives. Before relying on it, verify that your chosen model and API endpoint produce consistent results on a few known-good and known-bad packages, and decide whether AUDIT_FAILURE_FATAL=false is acceptable for your workflow. The tool is young, has no tagged releases, and its license is unstated, so check those details before integrating it into a critical path.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository