serve-sim: Stream an Apple Simulator to the Browser for AI Agent Use

What serve-sim Does and Why It Exists

iOS Simulators run locally on macOS. AI coding agent tools that can interact with a browser or a web interface cannot directly see or control a simulator running in Xcode. serve-sim bridges that gap: it captures the simulator framebuffer, streams it as MJPEG to a browser, and forwards input events back to the simulator through a WebSocket control channel.

The README describes the primary use case as hosting the simulator for agent tools like Codex, Cursor, or Claude Desktop. A secondary use case is remote testing: the served URL can be tunneled over a network so that someone on another machine can see and interact with the simulator as if it were running locally.

The README notes that the author develops the Expo framework but that serve-sim is completely agnostic to React Native and can be used for any iOS interaction. It works with any booted iOS Simulator without requiring a Xcode plugin or any instrumentation in the app being tested.

The last push was on 2026-09-26. The repository is not archived.

How serve-sim Works: Swift Helper, MJPEG, and WebSocket

When you run npx serve-sim, it spawns a small Swift helper that captures the simulator's framebuffer using xcrun simctl io. The helper converts the framebuffer into an MJPEG stream. A WebSocket control channel sits alongside the stream and accepts input events (gestures, keyboard input, button presses) that the tool translates into simulator commands.

A React preview UI is served on top of these streams at http://localhost:3200. The browser receives the MJPEG frames at up to 60 FPS and sends user interactions back through the WebSocket.

The Swift helper is compiled as an arm64 binary and ships with the npm package as serve-sim-bin. This is why the README explicitly states that serve-sim runs on Apple Silicon only and does not support Intel (x86_64) Macs. The helper uses private or semi-private simulator APIs to access the framebuffer and inject events.

For camera injection, a separate host-side helper writes BGRA frames into a POSIX shared-memory region, and an injected dylib (DYLD_INSERT_LIBRARIES) swizzles AVFoundation inside the simulator process so the app reads from that shared memory region instead of the stub camera.

Starting serve-sim and Basic CLI Commands

The simplest way to start serve-sim is:

npx serve-sim

This auto-detects any booted simulator and opens a preview at http://localhost:3200. To target a specific device:

serve-sim "iPhone 16 Pro"

To start a background helper and return JSON with the stream details:

serve-sim --detach

To list running streams:

serve-sim --list

To stop all helpers:

serve-sim --kill

Text input goes directly to the focused field in the simulator:

serve-sim type "Hello, world!"

Multiple simulators can be attached simultaneously. Pass several device names, or leave the argument empty to attach to all booted simulators.

Simulator Controls: Gestures, Keyboard, and Xcode 27 Notes

The browser preview forwards keyboard commands and hotkeys to the simulator, including CMD+SHIFT+H to go home. The preview UI supports swipe-from-bottom to go home and pinch-to-zoom by holding the option key. Simulator logs are forwarded to the browser, which allows browser-use MCP tools to read them.

For Xcode 27, keyboard input uses a different path through Device Hub. The README states that Device Hub must be running with the target simulator window visible and frontmost. The app that launched serve-sim (for example Terminal) may also need Accessibility permission in System Settings. Xcode 26 and older use the legacy HID path automatically.

If input stops working after a Device Hub reconnect, the repair command is:

serve-sim repair-input -d <udid>

The README warns that this command restarts SpringBoard and closes all running apps. It is never run automatically by serve-sim itself; you must run it explicitly. After running it, restart serve-sim to reconnect guest HID services, then reopen your app.

The environment variable SERVE_SIM_DISABLE_DEVICE_HUB_KEYBOARD=1 opts out of the Xcode 27 bridge entirely, falling back to the legacy path.

iPhone Duo Support for Xcode 27.1 Foldable Simulators

With Xcode 27.1 and the iOS 27.1 runtime, serve-sim exposes controls for iPhone Duo, a foldable device simulator. The browser preview gains Folded, Semi-folded, and Fully open controls. Selecting a pose drives the simulator's hinge and follows the active panel's framebuffer.

Hinge angles can be set from the CLI:

serve-sim fold 0 -d <udid>
serve-sim fold 180 -d <udid>
serve-sim fold 130 -d <udid>

Commands wait for the simulator to acknowledge the angle. Invalid angles and failed commands do not change capture state. The bundled simduo/serve-sim-duo-hid executable runs inside the simulator and exits with the device session.

The README notes that Duo relies on private beta APIs and the selected Xcode's V68.usdz model, so compatibility must be rechecked after SDK updates. The preview follows hinge changes made in Device Hub as well as changes made through the CLI.

Requirements, Limitations, and What serve-sim Cannot Do

The README lists three hard requirements: macOS with Xcode command-line tools (xcrun simctl), a maintained Node.js LTS release (currently Node.js 20 or later), and Apple Silicon hardware. Older or end-of-life Node versions are not supported. bun is not required for the CLI.

serve-sim does not run on Intel Macs. The serve-sim-bin helper ships as an arm64 binary only.

serve-sim works with iOS Simulators. The README does not describe support for real device streaming. It also does not describe support for Android emulators.

For the Xcode 27 keyboard path, the streaming cannot start if Device Hub is stuck on Connecting display and xcrun simctl io enumerate lists no framebuffer ports. The README says streaming cannot start until the display connection is restored, and there is no automatic recovery path for this state.

Camera injection uses DYLD_INSERT_LIBRARIES, which swizzles AVFoundation inside the simulator process. This approach is specific to the iOS Simulator environment and cannot be used with real devices.

How serve-sim Compares to Using xcrun simctl Directly

xcrun simctl is the Apple command-line tool for managing simulators. It can screenshot, record video, install apps, set permissions, and trigger push notifications. An AI agent with shell access could use xcrun simctl directly for discrete automation tasks.

serve-sim differs in two ways. First, it provides a continuous live stream rather than individual screenshots, which is more useful for agents that need to observe ongoing state changes. Second, it provides a browser-accessible interface that agents communicating through browser-use tools or MCP browser tools can read without needing shell access.

For a developer who wants to run discrete scripted tests against a simulator, xcrun simctl is sufficient. For an AI agent that needs to observe the simulator continuously and respond to UI state changes as they happen, serve-sim's persistent stream is the more practical interface.

Editorial conclusion

serve-sim is the right tool for iOS developers who work with AI coding agents and need those agents to see and interact with a running simulator. It is not a substitute for Xcode's own simulator interface, and it does not run on Intel Macs. Before using it with Xcode 27, read the Device Hub keyboard input note in the README: input may stop working after a Device Hub reconnect, and the repair command (serve-sim repair-input) restarts SpringBoard and closes running apps.

Frequently asked questions

Official sources

1. EvanBacon/serve-sim on GitHub
2. Issues
3. License: Apache-2.0
4. README