Peekaboo: A macOS CLI and MCP Server That Lets AI Agents See and Click the Screen

What Peekaboo Solves and Who It Serves

Peekaboo addresses a gap in macOS automation: giving AI agents a reliable way to see the screen and act on it. The README describes it as a CLI and menu-bar app for screen capture, accessibility inspection, and native UI automation. The target user is a developer building or using AI agents that need to operate macOS applications, whether through a natural-language agent loop or through MCP clients like Codex, Claude Code, or Cursor. It is not a general-purpose screenshot tool; it is a bridge between an agent's intent and the concrete UI elements on screen. The project also spawns community rewrites for Windows and Linux, which suggests the core loop is portable, but this version is macOS-only.

The Core Loop: Observe, Map, Act

The central mechanism is a three-step cycle. First, you capture the screen or a specific app window with commands like `see` or `window list`. The output can be JSON, which gives you a structured UI map with opaque element IDs. Second, you choose an element from that map. Third, you act on it using commands such as `click`, `type`, `press`, `scroll`, or `drag`. The README shows this clearly: `peekaboo see --app Finder --json` produces a UI map, then you can run `peekaboo click "Address and search bar" --app Safari --window-id 12345`. The element IDs are opaque, meaning you must use the output as the source of truth rather than guessing. This design avoids brittle coordinate-based automation, at least when the accessibility tree is available.

Installation and Permissions: The Real Gate

Getting Peekaboo running is straightforward but has hard requirements. The released CLI and app require macOS 15 or later. The Homebrew route is `brew install steipete/tap/peekaboo`. For MCP integration, you need Node.js 22 or later and run `npx -y @steipete/peekaboo --version`. The menu-bar app is a separate signed DMG download. Permissions are the bottleneck. Screen capture needs Screen Recording permission, and UI inspection and control need Accessibility permission. The README mentions an additional permission for synthetic input, which likely covers sending keystrokes or mouse events. There is a `peekaboo permissions status` command to check what is available. If you cannot grant these permissions, the tool is effectively unusable, so this is a deployment consideration, not a minor setup detail.

Background Automation and Its Constraints

Peekaboo supports background input delivery when it can resolve the target process, meaning the app does not have to become frontmost. This is useful for automation that does not disrupt the user's current focus. However, the policy is strict. The README states that app/PID-only and targetless chords require explicit foreground consent, as do window-selector-only Agent/MCP chords. A fresh exact non-dialog snapshot is required for background-only Agent/MCP policy. This means you cannot simply say "click the button" without specifying a window or providing a snapshot in an automated context. The design prioritizes safety over convenience, which is sensible for a tool that can control the UI, but it adds friction. You must carefully pass window IDs or snapshots to stay in the background path.

Agent and MCP: Two Ways to Let AI Drive

Peekaboo offers two integration styles. The first is a built-in agent: `peekaboo agent "Open Safari, go to github.com, and search for Peekaboo" --allow-foreground`. This requires a configured model provider, stored under `~/.peekaboo`. The second is exposing the same tools to an external MCP client via the `mcp` command. The README points to docs for connecting to Codex, Claude Code, or Cursor. The agent combines observation and action tools into a natural-language run, which is a higher-level abstraction than the raw CLI. The `--allow-foreground` flag suggests that foreground actions are gated by default. This is a meaningful design choice: the agent cannot perform actions that require the app to come to the front unless you explicitly permit it, which is a guardrail against unintended UI disruption.

Limitations and Failure Modes

The most obvious limitation is platform lock-in. Peekaboo is macOS-only, and the README points to community rewrites for Windows and Linux, but those are separate projects with different codebases. Within macOS, the tool depends on the accessibility API. If an app does not expose its UI elements through accessibility, the structured map will be incomplete, and coordinate-based fallbacks may be needed, which the README does not detail. The permission model is another failure point: without Screen Recording or Accessibility, commands fail or produce empty results. The 4.2.3 release notes mention hardening around credentials, which implies that earlier versions had credential exposure risks. The note about "locked-session" recovery suggests that automation can break when the screen is locked, which is a real operational edge case. The tool also requires a model provider for agent mode; if you have no provider configured, the agent command will not run.

Alternatives and How They Differ

The main alternative is Apple's own accessibility and UI testing frameworks, such as XCTest for UI testing. XCTest is designed for automated tests, not for AI agents, and it requires an Xcode project and a test runner. Peekaboo is a standalone CLI that can be invoked from any agent, which is a different integration model. Another alternative is using raw AppleScript or System Events via `osascript` to control apps. That approach is scriptable but lacks the structured JSON output and the MCP server integration that Peekaboo provides. AppleScript is also less reliable for modern SwiftUI apps, whereas Peekaboo claims to support verified non-modal SwiftUI windows in background automation. The key difference is that Peekaboo is built as a tool for AI agents, with a permission model and a policy for background actions, whereas the alternatives are either test frameworks or lower-level scripting languages.

Maintenance and License Considerations

Peekaboo is under the MIT license, which is permissive and allows commercial use without copyleft obligations. The repository is actively maintained, with recent releases in August 2026, including patch updates like v4.2.2. The release notes show a pattern of hardening: credential handling, OAuth, clipboard, and window recovery. This suggests ongoing maintenance, but it also means you should track releases for security fixes, especially given the credential-related changes. The project uses Swift 6.2 and Node.js 22 for development, and the build process uses pnpm with commands like `pnpm run build:cli`. Upgrading likely requires rebuilding if you install from source, but Homebrew and npm packages handle updates separately. The documentation is extensive, with guides for permissions, automation, and MCP, which reduces the learning curve but also implies that the tool has many configuration options that you need to understand to use effectively.

Editorial conclusion

Adopt Peekaboo if you build AI agents on macOS that must interact with native apps and you can meet the macOS 15 and permission requirements. Avoid it if you need cross-platform automation or you cannot grant Screen Recording and Accessibility permissions in your environment. Before adopting, verify that the specific UI elements you target are exposed via accessibility and that your model provider works with the agent or MCP configuration. Confirm the background automation policy fits your use case, since app/PID-only actions require explicit foreground consent.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes