Ghost OS Review: Accessibility-Tree Computer Use for AI Agents on macOS

What Ghost OS solves, and who it is actually for

A coding agent can edit files and run tests, but it cannot click Send in a mail client or drag a file between Finder windows. Ghost OS exists to close that gap on macOS. The README frames the problem plainly: the agent "lives inside a chat box." Ghost OS is the layer that gives it eyes and hands on the desktop.

The audience is narrow and specific. You need an MCP-capable client (Claude Code, Cursor, VS Code, or anything else speaking the Model Context Protocol), a Mac running macOS 14 or later, and a workflow that ends in a native application rather than a terminal. Teams that only automate HTTP APIs will get nothing from it. The project is written in Swift 6.2 and published under MIT, so the source is readable if you want to audit what an agent can reach.

There is a second audience the README is clearly courting: people who want to hand a repetitive desktop task to a small model instead of paying a frontier model to re-derive the same clicks every run. That is the recipe system, and it is the part worth evaluating.

Accessibility tree first, ShowUI-2B as the fallback

The core design choice is what the agent looks at. Screenshot-based computer use sends pixels to a vision model and asks it to guess where the button is. Ghost OS instead reads the macOS accessibility tree, which exposes structured, labeled data about every element in every app. A button has a role, a label, and a position. That is cheaper and more stable than pixel inference.

The README is honest that this does not cover everything. When the AX tree is not enough, specifically web apps and dynamic content, Ghost OS falls back to a local vision model, ShowUI-2B, for visual grounding. So the architecture is two-tier: structured data when the app cooperates, a local model when it does not. Nothing in that path leaves the machine, which is the basis for the local-data claim.

The recipe layer sits on top. During a learning session, Ghost OS observes actions through a CGEvent tap enriched with accessibility tree context. The README says Claude then synthesizes the raw observation into a parameterized, replayable recipe. At run time, ghost_run executes that recipe with parameters. The economics are the point: a frontier model reasons through the workflow once, and a small model replays it thereafter. Recipes are JSON, which means you can read every step before executing it.

Installing Ghost OS and recording your first recipe

The documented install is two commands. The first pulls the formula from the project's Homebrew tap, the second runs the setup wizard, which the README says handles permissions, MCP configuration, recipe installation, and vision model setup.

brew install ghostwright/ghost-os/ghost-os
ghost setup

The README warns about one failure mode here: Homebrew has a known issue on macOS developer betas where it demands an Xcode version that does not exist yet. If brew install fails on a beta, the README gives a manual path that downloads the arm64 tarball from the latest release and copies the ghost and ghost-vision binaries into /opt/homebrew/bin/.

curl -sL https://github.com/ghostwright/ghost-os/releases/latest/download/ghost-os-2.2.1-macos-arm64.tar.gz | tar xz
sudo cp ghost /opt/homebrew/bin/
sudo cp ghost-vision /opt/homebrew/bin/

Recording a workflow uses three MCP tools. ghost_learn_start begins watching, ghost_learn_stop returns the enriched action sequence, and ghost_learn_status reports progress. The README's example: you tell the agent to watch you send an email, you perform the task manually, and ghost_learn_stop returns eight actions with full AX context. The agent synthesizes a recipe with three parameters (recipient, subject, body) and saves it with ghost_recipe_save. Later, ghost_run recipe:"gmail-send-learned" params:{...} replays it. Recording requires Input Monitoring permission under System Settings > Privacy & Security > Input Monitoring.

Where Ghost OS breaks, and when it is the wrong tool

The platform boundary is absolute. The badge says macOS 14+, the description says native macOS, and nothing in the README suggests a Windows or Linux port. If your fleet is mixed, this covers one slice of it.

Permission friction is the second limit. Learning requires Input Monitoring, which is a system-level grant a user has to make by hand. On a managed machine, that may be a policy conversation rather than a checkbox. The README points at ghost setup to configure it, but the grant itself is manual.

The deeper fragility is inherent to the approach. An accessibility-tree-driven recipe encodes assumptions about the element structure of a specific app version. Gmail's compose window is not a stable contract. The README itself names web apps and dynamic content as the cases where the AX tree is insufficient, which is exactly where a recorded recipe is most likely to drift. There is no documented mechanism for detecting that a recipe has gone stale, and the README does not document rollback. Treat a learned recipe as something to re-verify after app updates, not as a durable artifact.

Finally, if the task is browser-only, a DOM-driven tool is a better fit. Ghost OS is built for the case where the workflow leaves the browser.

Ghost OS compared with Anthropic Computer Use and browser DOM tools

The README's own comparison table puts Ghost OS against Anthropic Computer Use, OpenAI Operator, and OpenClaw. The differences in approach are real, not cosmetic.

Anthropic Computer Use sees through screenshots only. That works on any app and any platform, because pixels are universal, but every step costs a vision model call and the agent re-reasons the workflow on each run. Ghost OS trades that portability for structure: the accessibility tree gives labeled elements, and a recorded recipe replaces repeated reasoning with replay. The trade is that Ghost OS only runs on macOS, and only where the AX tree is informative.

OpenAI Operator is browser-only and cloud-hosted, so the data question is different by construction. OpenClaw reads the browser DOM, which is more precise than pixels inside a page but stops at the browser boundary. Ghost OS operates Slack, Finder, and Messages, which neither of those two reaches. That is the actual differentiator: native app coverage on one platform, versus browser coverage on many.

Licensing differs too. Ghost OS and OpenClaw are MIT; Anthropic Computer Use and OpenAI Operator are not open source. If you need to read the code that touches your desktop, that narrows the field to two.

Maintenance, upgrade cost and the MIT licence

The last push to the repository was on 2026-03-23, and the most recent release is v2.2.1 from 2026-03-12. The release cadence around that date was tight: v2.1.2 on 2026-03-11, then v2.2.0 and v2.2.1 on 2026-03-12. Since then there has been no push, so the project is not under active development as of the last recorded activity. Plan around the version you install rather than around a stream of fixes.

The upgrade surface is small. Ghost OS installs through a Homebrew tap, so brew upgrade handles the binary, but ghost setup is what wires permissions, MCP configuration, recipes, and the vision model. After an upgrade, re-running ghost setup is the documented way to bring those pieces back into line. The manual install path pins a specific tarball (ghost-os-2.2.1-macos-arm64.tar.gz), which means beta users on that path have to change the URL by hand for each version.

On licensing: MIT is permissive, and the practical implication is that you can read, modify, and redistribute the Swift source. That is a statement about the licence text, not advice about your situation. If you intend to ship Ghost OS inside a product, have someone qualified review the terms and the bundled vision-sidecar directory, which is part of the repository layout.

Editorial conclusion

Adopt Ghost OS if your agents already live in an MCP client and the work happens in native macOS apps where the accessibility tree is well populated. Skip it if you need Windows or Linux, if the target is a browser-only workflow where a DOM-driven tool is a better fit, or if you cannot grant Input Monitoring. Before relying on it, record one recipe with ghost_learn_start and ghost_learn_stop, inspect the generated JSON, and confirm it replays after the target app updates.

Frequently asked questions

Official sources

1. ghostwright/ghost-os on GitHub
2. License: MIT
3. Project website
4. README
5. Releases