Peekaboo: a macOS CLI and MCP server for screen capture and UI automation

What Peekaboo solves for macOS automation

Most scripting against a Mac stops at the point where you need to know what is on screen. AppleScript and shell tools can launch an app, but they do not give you a structured description of the current interface, and they cannot reliably answer whether a button exists before you try to press it. Peekaboo fills that gap by combining three things in one binary: screen capture, accessibility inspection, and synthetic input. The README describes it as "a macOS CLI and menu-bar app for screen capture, accessibility inspection, and native UI automation".

The intended audience is narrow and specific. It is for engineers wiring an AI agent to a Mac, and for anyone who wants a scriptable way to drive a GUI application that has no API. The MCP server is the part that matters for the first group: the same toolset the CLI exposes can be handed to Codex, Claude Code, Cursor, or another MCP client, so the agent does not need a bespoke integration per application. The second group gets a command line tool that can list windows, click a named element, and type text without writing Swift.

What it is not: a cross-platform automation layer. The released CLI and app require macOS 15 or later, and the README instead links to PeekabooWin (a Windows-first rewrite in JavaScript and PowerShell) and PeekabooX (a Linux-first rewrite in Rust and Python) for other operating systems. Those are separate projects, not ports maintained inside this repository.

The observe, choose, act loop and how targeting works

The core mechanism is a three-step loop the README states plainly: "observe the current screen, choose an element from the result, and act on it." Observation comes from `see`, which can capture the whole screen or a single app and, when accessibility data is available, returns a structured UI map with opaque element IDs. Action comes from commands like `click`, `type`, `press`, `scroll`, `drag`, `set-value` and `action`.

The interesting design decision is how actions are targeted. Peekaboo can address an element by its semantic name, by an element ID from a snapshot, or by coordinates, and it can pin an entire interaction to a specific window with `--window-id`. The README states that targeted semantic and typed CLI input uses background delivery when Peekaboo can resolve the process, so the target app does not have to become frontmost. That matters on a machine you are also using, because stealing focus mid-script breaks whatever the human was doing.

The policy layer is where the trade-offs show. Background delivery is not unconditional. According to the README, raw CLI `press` chords can stay in the background with an exact window selector, and the CLI also accepts "a fresh exact non-dialog snapshot" that background-only Agent and MCP policy requires. App/PID-only and targetless chords need explicit foreground consent, and so do window-selector-only Agent and MCP chords. The README advises preferring a semantic action such as `menu click` when one exists. Read that as a deliberate restriction: the agent-facing surface is smaller than the CLI surface, and the difference is enforced rather than documented as a suggestion.

Installing Peekaboo and taking a first screenshot

There are two install paths, and they are not interchangeable. The Homebrew formula installs the CLI. The npm package bundles the CLI plus its MCP launcher and needs Node.js 22 or later. The README gives this command for the tap:

brew install steipete/tap/peekaboo

If you would rather not install globally, the npm package can be run without a permanent install. The README shows this version check:

npx -y @steipete/peekaboo --version

The menu-bar app is a third artifact. It is downloaded as a signed DMG from the latest GitHub release and provides permission onboarding, visual feedback, and agent sessions. The README notes that you install the CLI separately when you also need `peekaboo` on `PATH`, so the DMG does not replace the Homebrew or npm install.

Before capturing anything, check what the process is actually allowed to do. The README's quick start begins here:

peekaboo permissions status
peekaboo see --no-elements --mode screen --path /tmp/peekaboo-screen.png

Screen capture needs Screen Recording permission. Accessibility permission is what enables UI inspection and control, and the permissions guide covers an additional permission used for synthetic input. If `permissions status` reports a gap, the screenshot command will fail or return an empty result rather than a helpful error, which is the first thing to rule out when something looks broken.

The next step up is a structured inspection of a running application. This returns a UI map with element IDs you can act on later:

peekaboo see --app Finder --json

For a full interaction, list the target app's windows, copy the `window_id` you want, and keep every subsequent command pinned to it:

peekaboo window list --app Safari --json
peekaboo click "Address and search bar" --app Safari --window-id 12345
peekaboo type "github.com/openclaw/Peekaboo" --app Safari --window-id 12345
peekaboo press Return --app Safari --window-id 12345

The `12345` is a placeholder for the ID you read from the window list, not a fixed value. Run `peekaboo help <command>` for live flag help; the README points at the complete command index for anything not covered there.

The agent and MCP layer, and its model dependency

The agent command folds observation and action into a single natural-language run. The README's example is:

peekaboo agent "Open Safari, go to github.com, and search for Peekaboo" --allow-foreground

That flag is not decoration. Agent runs need a configured model provider, and the README's own example passes `--allow-foreground` because the run may need to bring windows forward. Provider credentials and settings live under `~/.peekaboo`, managed with `peekaboo config`. The configuration guide covers profiles, environment variables and custom providers, and the provider reference covers hosted, compatible and local model backends.

So there are two distinct costs here. One is the permission model. The other is that the agent path is only as capable as the model you point it at, and you have to configure that before the agent command is useful. The CLI capture and click commands do not need a provider; the agent does. If you are evaluating Peekaboo purely as a screenshot and automation tool, you can ignore the agent entirely and never touch `~/.peekaboo`.

The MCP route is the same toolset exposed to another client. The README directs you to the MCP setup guide to connect it to Codex, Claude Code, Cursor, or another MCP client, and the npm package includes the launcher. Note the policy asymmetry again: the README states that background-only Agent and MCP policy requires a fresh exact non-dialog snapshot, which is a stricter requirement than the CLI's. An MCP client that passes a stale snapshot will be refused.

Where Peekaboo breaks down or is the wrong choice

The hardest constraint is the operating system. macOS 15 or later for the released CLI and app, with no fallback documented for earlier versions. If your fleet is on macOS 14, this is not a configuration problem you can work around. The README's answer is to look at the separate Windows and Linux rewrites, which are different codebases with different command surfaces, so any automation you write against Peekaboo's CLI does not carry over.

Permissions are the second failure mode, and they are not self-diagnosing. Screen Recording, Accessibility, and the additional permission for synthetic input are granted per process and can be revoked by the user or by an OS update. The README documents a `permissions status` command and a permissions guide, but the failure you will actually hit is a command that returns nothing useful because a permission was silently dropped. On a headless or locked machine this gets worse: the 4.2.3 release notes mention "locked-session" recovery, which tells you that a locked session is a real state the tool has to handle rather than a case it excludes.

The third limitation is targeting fragility. Element names come from the accessibility tree, and apps that draw custom controls or render their UI in a web view may expose nothing meaningful. That is why the CLI also accepts coordinates and fresh snapshots. Coordinates are brittle across window resizes and display changes; snapshots go stale. The README's advice to prefer semantic actions such as `menu click` is the right default, but it only applies where the accessibility tree cooperates. If your target app exposes a poor tree, you are back to pixel positions, and the reliability drops accordingly.

How Peekaboo differs from a screenshot-only tool

The obvious alternative is a plain screenshot utility plus a separate automation library, for example `screencapture` on macOS combined with AppleScript or a Python accessibility binding. The difference in approach is that Peekaboo keeps observation and action in one process with one permission set, and returns structured element IDs rather than pixels you have to interpret. With `screencapture` you get an image and nothing else; deciding where to click is your problem. With `peekaboo see --app Finder --json` you get a UI map, and the same element IDs are accepted by the action commands.

The second alternative is an MCP client's own computer-use tooling. The difference is scope: Peekaboo is a standalone CLI first, so you can script it in a shell, run it from CI, or call it from a language that never speaks MCP. The MCP server is an additional surface on the same binary, not the only way in. That is a meaningful distinction if you want to test the automation without an agent in the loop, because you can run the exact commands the agent would run and see what they return.

What Peekaboo does not give you is a recording or replay layer, a scheduling system, or cross-platform parity. It is a tool that observes and acts on the current state of a Mac, and it expects you to build the orchestration around it.

Maintenance, packaging and licence cost

The last push to the default branch was on 2026-08-20, the same day as the v4.2.2 release, with v4.2.0 and v4.1.0 landing earlier that month. That is a compressed release cadence, and the README's "What's new in 4.2.3" section describes hardening across Gemini, OAuth, clipboard and editor workflows plus changes to window inventory and background automation. Frequent point releases mean you should pin a version rather than track the tap or the npm tag, because the command surface is still moving.

There is a version discrepancy worth checking before you deploy. The repository's package.json declares version 4.4.0, while the newest release listed in the repository is v4.2.3 and the newest tagged release note is v4.2.2. The README does not explain the gap. Confirm which artifact you are actually installing, especially if you are mixing the Homebrew CLI with the npm MCP launcher.

Building from source is heavier than installing. The README lists macOS 15 or later, Swift 6.2 or later, Node.js 22 or later, and the repository's submodules as prerequisites, with pnpm install, a CLI build, a docs lint and a safe test target. The repository layout confirms the submodule dependency: top-level entries include AXorcist, Commander, Swiftdansi, Tachikoma and TauTUI alongside Core, Apps and Helpers. If you only need the tool, use the packaged installs and skip the source build entirely.

The licence is MIT, which permits commercial use and modification with the copyright notice retained. That is the whole of the licence implication here; nothing in the README describes a separate commercial tier, a hosted service, or an enterprise licence. Note that the model providers you configure under `~/.peekaboo` have their own terms, and Peekaboo's MIT licence says nothing about those.

Editorial conclusion

Adopt Peekaboo if you are automating macOS apps on machines running macOS 15 or later and you already accept the Screen Recording and Accessibility permission model; the MCP path is the reason to pick it over a plain screenshot utility. Do not adopt it if you need Windows or Linux coverage, since the README points those users at the separate PeekabooWin and PeekabooX rewrites, or if you cannot grant synthetic input permissions on the target host. Verify first that `peekaboo permissions status` reports what you expect on the exact machine, and confirm the CLI version you install matches the release you intend to pin, because the npm package.json lists 4.4.0 while the newest release note in the repository is 4.2.3.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes