# phone-harness: letting a coding agent drive a real iPhone or Android

> phone-harness connects Claude Code, Codex or another agent to a physical phone, using iPhone Mirroring on macOS and adb on Android. It is a small, alpha-stage Python tool, and the interesting part is how narrow its mechanism is.

**ShawnPana/phone-harness** — let your agent control your phone

- Repository: https://github.com/ShawnPana/phone-harness
- Stars: 3,125 · Forks: 323
- Language: Python
- License: MIT
- Published: 2026-09-10 · Updated: 2026-09-10 · Language: en
- Canonical page: https://hysenlabs.com/projects/shawnpana-phone-harness

## The gap phone-harness fills: an agent with hands on a real device

Most agent tooling stops at the browser. A coding agent can read a page, fill a form and click a button, but a native mobile app is a closed surface: no DOM, no headless mode, no scriptable entry point. phone-harness takes the opposite route from emulators and device farms. It drives the phone you already own, through interfaces the operating system already exposes.

The README states the constraint set plainly: iPhone through the Mac's iPhone Mirroring window, Android over adb, with no jailbreak, no Xcode, and nothing installed on the phone. That last clause is the design centre. There is no companion app to sideload, no instrumentation framework, no accessibility service to enable on iOS. The tool is a Python package that runs on the host machine and treats the phone as a remote display and input target.

The intended user is someone who already has an agent in the loop, Claude Code or Codex, and wants it to complete a task that ends on a phone. The README's own demo task is a ride-hailing booking, which is a useful illustration of the shape: a short sequence of taps and text entry against an app that has no API for the purpose.

## How the iPhone path works: a mirrored window, Vision OCR, and HID events

iPhone Mirroring renders the phone as a macOS window and forwards mouse and keyboard input as touches. phone-harness sits on top of that window. It captures the window contents, runs OCR using Apple's Vision framework, and gets back text plus tap-ready coordinates. It then posts HID-level events for taps, swipes and typing.

The consequence is that the agent's perception is text, not pixels in the general case. `find_text("Weather")` returns a coordinate pair, `tap(400, 468)` acts on it, and the next capture tells you what happened. The README's flow diagram shows exactly this loop. That is cheap and fast when the target is a labelled button or a list row, and it is the reason the tool can run without a vision model at all.

The dependencies in pyproject.toml match the mechanism: pyobjc-framework-Quartz, pyobjc-framework-Vision, pyobjc-framework-Cocoa and pyobjc-framework-ApplicationServices. Those are the Quartz capture and event APIs and the Vision OCR bindings. The package metadata also lists `Environment :: MacOS X` and `Operating System :: MacOS`, and the description reads "Control a real iPhone through macOS iPhone Mirroring: OCR eyes, CGEvent hands". The iPhone path is a macOS-only feature, not a cross-platform one.

The Android path swaps every layer. adb is the transport, `screencap` is the capture, the phone's accessibility tree is the text source, and `input` is the hands. It works over USB or Wi-Fi and needs no window. The helpers are the same on both platforms, and `phone-harness config set platform ios|android` picks the default. That shared surface is the strongest part of the design: an agent skill written against the helpers does not have to care which phone is attached.

## Installing phone-harness and running a first script

The README does not give a pip command. It gives a setup prompt to paste into Claude Code or Codex, and the prompt does the work: it clones the repository into `~/.phone-harness`, reads `install.md` first, installs the tool so `phone-harness` is a command on your PATH, and registers it as an agent skill named phone-harness using `phone-harness skill` as the body. It then reads `onboarding.md` and walks you through it.

```text
Set up phone-harness for me. Clone https://github.com/ShawnPana/phone-harness into ~/.phone-harness, read `install.md` first, install it so `phone-harness` is a command on my PATH, and register it as an agent skill named phone-harness using `phone-harness skill` as the body. Then read `onboarding.md` and walk me through it.
```

The agent asks which phone is your default and walks you through the parts that need your hands: pairing iPhone Mirroring and granting Accessibility and Screen Recording, or turning on Android developer options and approving adb. Those permissions are granted by you in macOS System Settings; the tool cannot grant them. The README points to install.md for details.

Once the chain is up, `phone-harness --doctor` checks it. That is the command to run before anything else, because a missing permission fails silently otherwise.

```bash
phone-harness --doctor
```

Usage is a heredoc of Python. Helpers are pre-imported, so you do not import anything yourself. The README's example opens Notes, taps New Note, types a string, and prints the first ten OCR results.

```bash
phone-harness <<'PY'
open_app("Notes")
tap_text("New Note")
type_text("hello from the harness")
print([o["text"] for o in ocr()][:10])
PY
```

What you should see is the note appearing on the mirrored phone and a list of recognised strings printed back. `SKILL.md` is described as the agent's day-to-day guide, and `helpers.py` under `src/phone_harness/` is the full list of available functions. The skill text ships inside the wheel, so `phone-harness skill` works from an installed package and not only from a checkout.

## Where phone-harness breaks: OCR, multi-touch, and the locked phone

The README's Limits section is unusually honest, and three items matter more than the rest.

First, OCR sees text, not icons. An unlabelled control, a floating action button, a custom-drawn toggle, has no text for Vision to find, so `find_text` has nothing to return. The README's answer is to take a screenshot and hand it to a vision-capable model. That is a real fallback, but it changes the cost profile of the loop: you are now paying for image tokens on every ambiguous step, and the agent needs a model that can ground coordinates in an image.

Second, unlocking the iPhone pauses mirroring. A PIN-locked Android needs the user. Connecting the phone is always the user's job. There is no unattended mode here, and no credential handling that would create one. If your use case is a scheduled job that runs at 3am against a locked device, this is the wrong tool.

Third, there is no multi-touch, no camera and no Face ID flow, and DRM video renders black. The camera omission is structural rather than a missing feature: the harness only ever sees the mirrored window or the adb screencap, and a protected video surface does not render into either.

One more limitation is worth naming even though the README does not put it under Limits. The iPhone path inherits every property of iPhone Mirroring, including that it is a macOS feature. There is no Linux or Windows route to an iPhone here.

## phone-harness compared with Appium and device farms

The obvious alternative for mobile automation is Appium. The difference in approach is the whole story. Appium drives an app through platform automation APIs, which on iOS means WebDriverAgent and a signed test runner, and on Android means UiAutomator. You get element selectors, a WebDriver protocol, and language bindings across the ecosystem. You also get a build step, a signing story, and a runner process that has to be installed and kept alive.

phone-harness skips all of that by not automating the app at all. It automates the screen. On iOS it never touches the app's process; it reads a mirrored window and posts events at the HID level. That is why nothing is installed on the phone and why a jailbreak is not needed. The trade is precision. Appium can address an element by identifier and assert on its state. phone-harness can find the word "Weather" and tap the coordinate where it appears, which is a weaker contract and one that breaks when the layout shifts.

Hosted device clouds are the other comparison, and the README points at one: Phone Harness Cloud, described as hosted iPhones and Androids with stealth, real numbers, 2FA and unlimited devices. That is a different product with a different cost model, aimed at people who need many devices rather than the one in their pocket. The local tool and the hosted service are not substitutes; the README presents the cloud as an upsell from the open source package.

## Maintenance, packaging and the MIT licence

The repository is not archived, and the last push was on 2026-09-10, one week before this writing. Release 0.2.0 is tagged "0.2.0 (Android)" and dated 2026-08-18, with 0.1.0 the day before. The Android support appears to have landed in 0.2.0, which means the adb path is the newest and least exercised part of the codebase. The classifier still reads "Development Status :: 3 - Alpha".

Upgrade cost is low by construction. The package is pure Python with four pyobjc dependencies and a hatchling build backend. `requires-python` is ">=3.10". There is no compiled extension of its own, no service to migrate and no database. An upgrade is a reinstall, and the risk is in the pyobjc frameworks, which track macOS releases rather than the project's own schedule. Because the iPhone path depends on iPhone Mirroring, a macOS change to that feature is a change to this tool's foundation, and the project has no way to insulate itself from it.

The licence is MIT, declared in pyproject.toml via `license = "MIT"` and `license-files = ["LICENSE"]`, and the LICENSE file is present at the repository root. MIT is permissive: it allows commercial use and modification, and it requires that the copyright notice and permission notice be included in copies or substantial portions. That is a statement about the licence text, not legal advice; if you are redistributing the package inside a product, have your own counsel read the LICENSE file rather than this paragraph.

## Conclusion

Adopt phone-harness if you already run a coding agent on a Mac and want it to operate a real iPhone or Android app without installing anything on the device; the iPhone path in particular is only possible because of iPhone Mirroring, so a Mac is not optional. Do not adopt it for multi-touch games, camera or Face ID flows, or DRM video, all of which the README lists as out of scope, and do not expect it to unlock a phone for you. Before wiring it into anything, verify that `phone-harness --doctor` reports the whole chain as healthy on your machine, because the setup depends on macOS permissions (Accessibility and Screen Recording) that the tool cannot grant itself.

## FAQ

### What is a phone harness?

In this project, phone-harness is a Python command line tool that lets an agent such as Claude Code or Codex control a real phone. On iPhone it works through the Mac's iPhone Mirroring window; on Android it works over adb. Nothing is installed on the phone itself.

### What is phone harness?

It is a Python package named phone-harness that gives an agent eyes and hands on a real device: it captures the screen, reads text with OCR or the accessibility tree, and posts taps, swipes and typing. The same helpers work on iPhone and Android, and the default platform is set with phone-harness config set platform ios|android.

### Is a phone lanyard worth it?

The README does not discuss lanyards or physical accessories. phone-harness is a software tool for controlling a phone from a Mac or over adb, so this question is outside what the material covers.

### What is a phone strap called?

The README does not cover straps or any physical accessory. The project is a Python CLI for letting an agent drive a phone, so there is nothing in the documentation that answers this.

### What is the best smartphone harness?

The README does not compare phone-harness with other tools or rank them. It describes one approach: iPhone through the Mac's iPhone Mirroring window and Android over adb, with no jailbreak and nothing installed on the phone.

## Sources

- [Issues](https://github.com/ShawnPana/phone-harness/issues)
- [License: MIT](https://github.com/ShawnPana/phone-harness/blob/main/LICENSE)
- [README](https://github.com/ShawnPana/phone-harness/blob/main/README.md)
- [Releases](https://github.com/ShawnPana/phone-harness/releases)
- [ShawnPana/phone-harness on GitHub](https://github.com/ShawnPana/phone-harness)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/shawnpana-phone-harness