iCTRL: turning an iPhone into an endpoint an AI agent can drive

What the tool actually is

iCTRL is described as an AI-friendly iOS automation service built on XCUITest, and the first paragraph of the readme is the clearest statement of what it does.

It turns an iPhone into a controllable automation endpoint. It runs a long-lived XCUITest runner that exposes a JSON-RPC 2.0 API over HTTP, on default port 8200, so an AI agent or any HTTP client can drive the device: tap, swipe, type, read the UI tree, take screenshots, launch apps, open URLs and more.

The distinction that matters is between this and a raw XCUITest script. A script is a program you write and run. iCTRL is a service you point something at. The readme says it is designed to be discovered and driven by a model, and the mechanism for that is two HTTP endpoints. GET /skill serves a complete operating manual and GET /schema documents every method with its parameters, so an agent can learn the API at runtime and call it correctly without prior knowledge.

That is the design decision the whole project rests on. Instead of shipping a client library that an agent has to be given, the device describes itself. An agent that can make HTTP requests can use it, and the readme lists the ones it expects to work: OpenClaw, Hermes, Minis, Claude Code and Codex. All it needs is the phone's IP address and port, and these agents usually explore the endpoint and install the skill themselves to learn how to drive the phone. Minis is additionally optimized for iCTRL, so it carries out interactions on the phone faster.

The wiring-up story is designed around the same idea. Once the runner is up, with remote access enabled if the agent is not on the phone, you tell the agent one sentence: that your iPhone is running iCTRL at a given address and port, and to figure out how to drive it. It finds the manual at GET /skill and the method list at GET /schema, downloads a client from the device, and asks to pair. You approve the pairing notification on the phone, and from there it works out the rest.

The project is Swift, Apache 2.0 licensed, with the bundle identifier com.openminis.ictrl and the transport specified as JSON-RPC 2.0 over HTTP on port 8200. The recent releases are v1.98.12 from 2026-10-15 and v1.97.1 and v1.97.0 from 2026-10-13.

The security model, and the warning the readme leads with

The readme opens with a warning block, and it is unusually direct for a project whose whole purpose is to hand control of a device to something else.

For automated testing and research only. iCTRL hands full control of a phone, taps, text input, screen contents, every installed app, to whoever holds a paired key. Any automation like this can be hijacked or misused: a leaked key, a client you should not have approved, or an agent acting on instructions it picked up along the way.

Then the specific advice, and it reads like a checklist someone has actually been burned by. Run it only on devices you own and accounts you are entitled to automate. Approve only the pairing requests you started. Leave remote access off unless you need it, and stop the server when you are done. And do not leave an agent unsupervised around payments, messages or anything that can't be undone.

The closing line is that you are responsible for how you use it, including compliance with the terms of the apps you automate.

The authentication model itself is described compactly. The transport is JSON-RPC 2.0 over HTTP on port 8200, and the auth is an encrypted envelope, X25519 pairing leading to HMAC-SHA256 sealed requests. So the pairing is a key exchange rather than a shared secret typed into a prompt, and the sealed requests mean the traffic is not readable to anything sitting between the agent and the phone.

The visibility model is the other half, and it is exposed in the runner's notifications as well as in the companion app. The readme lists three levels: this device only, LAN, or LAN plus nearby through AWDL. The runner binds to 127.0.0.1 only by default, and Remote Access is what opens it to the LAN. An agent running on the phone itself, such as Minis, reaches the runner on 127.0.0.1 and needs nothing turned on. An agent on another machine needs Remote Access enabled and the same Wi-Fi network.

So the default posture is local-only, and widening it is a deliberate act with a visible notification. That is the right design for a tool of this kind, and the reason the readme tells you to stop the server when you are done rather than leaving it running.

Two apps, and what each one is for

The tool ships as two IPAs, and the readme is explicit about which is which and which one you actually need.

iCTRLUITests-Runner.ipa is the automation engine. It is required, and it is the one that must be running. iCTRL.ipa is the companion app, and it is optional.

The runner does all the automation and runs on its own. The companion app is for the surrounding chores on the phone itself, and the readme lists what it does.

It can mount the Developer Disk Image without a Mac, and re-mount it after a reboot in one tap. Pairing files can be picked in the app or shared to iCTRL from Files, Mail, AirDrop or a chat.

It can start and stop the runner, and see whether it is running and which version it is.

It gives visibility: you can see and switch who can reach the runner, between this device only, LAN, or LAN plus nearby through AWDL.

It lists authorized clients, every computer, agent and browser holding a key, where each last connected from, this device or USB, LAN, nearby or VPN, and you can revoke any of them.

And it shows runner logs live, with pause, refresh and filter.

The companion app pairs with the runner once like any client, with one Allow in the runner's notification. You can skip it entirely if you mount the DDI with Xcode or StikDebug and manage the runner from its notification or from a computer.

There is a practical constraint in that last sentence. With a free Apple ID's three-app limit, installing StikDebug or the iCTRL app is enough, but not both. So the choice of companion is partly a licensing arithmetic decision rather than a preference.

The prerequisites are the same either way. You need a way to sign the IPA, either a free Apple ID via SideStore or a paid account, and the Developer Disk Image mounted.

What the readme says about performance and speed

The features section is grouped under four headings, and the numbers are given with the honesty of a tool that has been benchmarked.

Under built for AI agents there are three items. Self-describing, which is the skill and schema endpoints, with the skill noted as under 15K characters with the most important rules first, which is a detail about what the agent reads first when it needs to prioritise. Responses that steer, where every result carries a hint and every error says what to do next, a ref re-resolves the live element at tap time, visibility and alternatives flag look-alikes, and scrolls report whether they moved and where you are in the list. And knows the apps on the phone, where resolve_app turns an app name into a bundle id offline and remembers which apps actually opened on this device, even where iOS refuses to list installed apps.

The fast heading gives the figures. Around 280ms taps, with synthesized touch events and no wait-for-idle. The visible screen in under a second, even on long-scrolled feeds, with a p50 of 416ms across 8 apps. And element lookups in roughly 0.3 to 1 seconds. The explanation for the last two is the same mechanism: snapshots are bounded on the device, so the app never serializes thousands of off-screen rows.

That bounding is the key decision in the performance story. The naive approach to reading a screen through the accessibility tree is to dump everything, and on a long feed that is thousands of rows and a multi-second response. Bounding the snapshot and doing the walk on the device turns the problem into a small fixed cost.

The third heading is the one that distinguishes this from a general UI automation library. Sees what the accessibility tree can't: on-device OCR through the screen_text method using Vision returns text with tappable coordinates, and page_source measures its own coverage and falls back to OCR automatically when the tree comes back thin, which is the case for web views and custom-drawn text. There is also feedback on coordinate taps, where a tap by coordinates reports which element it hit, or warns that it landed on empty space.

The fourth heading covers long, unattended sessions. It never queries an app that has left the screen, and such calls are refused in milliseconds instead of stalling for seconds, with system alerts and the home screen recognised as the foreground. The heading itself says the tool is diagnosable while stuck, though the readme is truncated at that point.

Building it, and what the repository contains

The Makefile shows that the build is a generated Xcode project rather than a committed one, which is a choice worth understanding.

A fresh clone has neither the project nor the signing config, and both are produced on demand, so a clone followed by make ipa works with nothing else set up. The project file is generated by xcodegen from project.yml, and the Makefile checks for xcodegen and fails with a message telling you to install it if it is missing. The local signing configuration is produced by a setup script, Config/Local.xcconfig.

The signing identity is read from the freshly built app rather than being written into the Makefile, and the comment is emphatic that it is never a guessed keychain identity. The re-sign therefore uses exactly what xcodebuild chose from the project's automatic signing, which is the detail that makes the whole flow work for anyone rather than only for the maintainer.

There is also a subtle build-order problem being solved in the Makefile, and the comment on it is a good piece of engineering. The build stamp is written before xcodebuild runs, not inside it. As an Xcode pre-build phase the generated file is written after the Swift compile has already read it, so the binary would ship the previous stamp, which is exactly the is my change deployed ambiguity the stamp is meant to remove.

The actual build uses xcodebuild build-for-testing with a derived data path, a scheme, a generic iOS destination, provisioning updates allowed, and ARCHS set to arm64.

The generated project and the build are two separate commands:

	xcodegen generate --quiet

	xcodebuild build-for-testing \
		-project $(PROJECT) \
		-derivedDataPath $(DERIVED_DATA) \
		-scheme $(SCHEME) \
		-destination "generic/platform=iOS" \
		-allowProvisioningUpdates \
		ARCHS=arm64

The repository layout shows the project is a set of Xcode targets rather than one app. There is iCTRLUITests/ for the runner, iCTRLApp/ for the companion, iCTRLKit/ and iCTRLCompat/ for shared support, cli/ for a command line component, Config/ for the signing configuration, Scripts/ for the helper scripts, and tools/. There is a SKILL.md at the top level, which is presumably the source of the operating manual the device serves at GET /skill.

There are also two Python scripts at the root, generate_app_bundles.py and generate_icon.py, and the project is described by project.yml, which is the xcodegen input. The top level also carries a NOTICE.md and a LICENSE, and the project name in the Makefile is iCTRL.xcodeproj with the scheme iCTRL.

Editorial conclusion

Adopt iCTRL if you want an agent to operate a physical iPhone you own, for testing or research, and you are prepared to handle the signing and pairing setup. Do not adopt it for production device fleets without reading the security notes, because the readme states plainly that the tool hands full control of a phone to whoever holds a paired key. Before you commit, read the warning section in full, keep remote access off unless you need it, and decide whether the companion app is worth installing, since the runner works without it.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. OpenMinis/iCTRL on GitHub
4. README
5. Releases