open-computer-use: an MCP server that drives macOS, Linux and Windows desktops through Accessibility

The gap it fills: Computer Use without a hosted vendor runtime

OpenAI's Codex Computer Use showed that a non-intrusive computer-use agent can be built on top of the operating system's Accessibility layer rather than on screenshots and coordinate guessing. The README says the author took that idea and built an open-source version. The result is a service, wrapped as an MCP server, that any MCP client can connect to and use to run Computer Use actions on macOS, Linux and Windows. The audience is narrow and specific: people who already have an agent (Codex App, Codex CLI, Gemini CLI, Claude Code, opencode) and want that agent to manipulate desktop applications, but who do not want that capability tied to one vendor's hosted runtime. The README's own framing is that it matches the official Codex experience when used from Codex App and Codex CLI. That claim is a demo description, not a benchmark, and the repository does not publish a compatibility matrix beyond the clients named in the install commands.

Accessibility as the control surface, MCP as the transport

The design choice that matters is the one inherited from Codex Computer Use: the runtime reads and drives applications through Accessibility rather than through a vision model looking at pixels. That is what makes it non-intrusive. The project ships as a Swift runtime with an npm distribution wrapper, and the MCP server is started with the subcommand open-computer-use mcp, which is exactly what the manual JSON config passes as args. Tools are addressed by name, for example list_apps and get_app_state, and results come back as MCP-style JSON, which the README shows being printed by the call subcommand. State between operations matters: the README notes that running a sequence in one process lets element_index state be reused, which implies that get_app_state returns indexed elements that later calls (press_key and similar) refer back to. That is the data flow. An agent asks for an application's state, receives an indexed element list, then issues an action against one of those indexes. Anything that breaks the index between calls, a UI redraw, a new window, a slow application, forces another state fetch. The README does not document how stale indexes are detected, so treat that as an open question rather than a solved one.

Installing it into an agent, and the config files it writes

The npm package is installed globally and also exposes ocu as a short alias. The README gives npm i -g open-computer-use as the install line. From there, the project does something more useful than printing a JSON snippet: it writes the MCP entry into each client's own config file. open-computer-use install-codex-mcp writes to ~/.codex/config.toml. open-computer-use install-claude-mcp writes to ~/.claude.json. open-computer-use install-gemini-mcp writes to ./.gemini/settings.json for the current project, and adding --scope user targets the user config instead. open-computer-use install-opencode-mcp writes to ~/.config/opencode/opencode.json, or the active config file. There is also install-codex-plugin, described as mainly for Codex App. If you would rather not use those commands, the README supplies the generic MCP block: a server named open-computer-use with command open-computer-use and args ["mcp"]. A separate skill is distributed through the skills CLI, with npx skills add iFurySt/open-codex-computer-use -g -a codex --skill open-computer-use -y for Codex and the same command with -a claude-code for Claude Code. Updates to that skill go through npx skills update open-computer-use -g -y. Because each install command edits a different file in a different format, the practical question before running them is which client you actually use; running several of them will leave MCP entries scattered across four config locations.

macOS 14 plus two permissions, and what happens when they are missing

The README is explicit that the macOS runtime requires macOS 14.0 or later, and that Windows and Linux do not need the permission step. On macOS you run the binary once and grant Accessibility and Screen Recording. That is the single largest operational constraint in the project. Screen Recording is a broad grant, and on managed machines it may be blocked by policy, in which case the tool has no path forward regardless of how the code is written. The doctor subcommand exists for this: open-computer-use doctor checks permissions, and the README notes that onboarding only opens when something is missing. That is a sensible design, since it avoids a permission prompt on every launch, but it also means a silent failure mode is possible if a permission is revoked after the first run. The README does not describe what error surfaces in that case. The other limitation worth naming is scope: this is a desktop automation tool, and the repository offers no evidence of a sandbox, an allowlist of applications, or an approval step before an action executes. Anyone pointing an autonomous agent at it is granting that agent the same desktop reach the process has.

The call subcommand, sequences, and why the CLI is not a scripting language

Beyond MCP, the CLI can invoke a single tool and print the result. open-computer-use call list_apps is the minimal case; open-computer-use call get_app_state --args '{"app":"TextEdit"}' passes arguments as JSON. For multi-step work there is a sequence form: open-computer-use call --calls with an inline JSON array of tool and args objects, or --calls-file pointing at a file such as examples/textedit-overlay-seq.json. The README states that sequence runs sleep 1s between successful operations by default, and that --sleep 0.5 changes it. This is the honest part of the design. Fixed sleeps are a blunt instrument: too short and the next action fires before the UI settles, too long and a twenty-step flow takes twenty seconds of dead time. There is no documented wait-for-condition primitive, so the sleep value is the only lever, and the right value depends on the machine and the application. The sequence runner exists mainly so element_index state survives across calls, which is the real reason to prefer it over firing separate call invocations.

Validation tooling, and what it tells you about the project's own testing

The README lists a set of local validation targets that run from a source checkout: make smoke, make stress with OPEN_COMPUTER_USE_STRESS_LOOPS=20, and make agent-smoke. The agent smoke tests are more interesting, because they drive real agents against the server. scripts/run-agent-smoke-tests.mjs takes --agents=claude,codex and --command=open-computer-use, with --scenario=fixture or --scenario=fixture-full. There is a Hermes path too, configured through OPEN_COMPUTER_USE_HERMES_PROVIDER and OPEN_COMPUTER_USE_HERMES_MODEL, with --hermes-max-turns controlling the loop. The existence of fixture scenarios suggests the tests run against a deterministic UI rather than whatever is on the developer's screen, which is the only way this kind of test can be stable. What the repository does not provide is any published result from these runs. There are no numbers attached to make stress, no pass rates for the fixture scenarios. So the tooling is there for you to run yourself, and the README gives the exact commands, but it is not evidence of anything until you execute it on your hardware.

Where it is the wrong tool, and what to use instead

If the task is web automation, this is the wrong layer. The README itself points readers to a sibling project, open-browser-use, for that case, and the split is meaningful: browser automation can work against a DOM and a network protocol, while open-computer-use works against the operating system's Accessibility tree and therefore inherits every quirk of native window management. Choose the browser tool when the target is a page; choose this one when the target is a desktop application that has no API. A second case where it is wrong: headless Linux. The README lists Linux as supported and says Linux does not need the macOS permission step, but a machine with no display session has no Accessibility tree to drive, and nothing in the supplied material describes a headless mode. A third boundary is unattended automation on a shared or managed Mac, where the Screen Recording grant is the blocker rather than the code. For those cases a scripted approach against an application's own CLI or API, or a hosted Computer Use service that runs in the vendor's own environment, avoids the permission problem entirely at the cost of sending the work off-machine.

Licence, release cadence and the cost of keeping up

The project is MIT licensed, which permits commercial use and modification provided the copyright notice and permission notice are retained; that is a statement about the licence text, not legal advice, and anyone embedding it in a product should read the file in the repository. The release history shows v0.3.5 on 2026-09-09, v0.3.4 the day before, and v0.3.3 about a week earlier. Patch releases one day apart indicate active churn in a pre-1.0 codebase. That has a concrete maintenance consequence: the install commands write into your agent's config files, so an upgrade can change what those files should contain, and the MCP entry is not automatically rewritten unless you rerun the install command. The skill has its own update path through npx skills update open-computer-use -g -y, which is separate from the npm package upgrade. Budget for two upgrade steps, not one, and expect the config format to move before version 1.0.

Editorial conclusion

Adopt it if you already run an MCP-capable agent (Codex, Claude Code, Gemini CLI, opencode) on macOS 14 or later and want desktop control without a vendor-hosted Computer Use service; the install commands write into each client's config file directly, which is the part that saves time. Do not adopt it if your target machines are headless Linux servers or you cannot grant Accessibility and Screen Recording, because the README states macOS needs both before anything runs. Verify first with open-computer-use doctor on the actual host, then run open-computer-use call list_apps and one call get_app_state against a throwaway app to confirm the element_index flow behaves before wiring it into an agent loop.

Official sources

1. iFurySt/open-codex-computer-use on GitHub
2. Issues
3. License: MIT
4. README
5. Releases