mobile-use: an LLM agent that drives Android and iOS through the accessibility tree
AI agents can now use real Android and iOS apps, just like a human.
At a glance
- What is it?
- Minitap's mobile-use turns natural language into taps on a real phone, splitting the work between a planner and an executor. It is Apache-2.0, Python 3.12+, and honest about where it fails.
- Who is it for?
- Adopt mobile-use if you are automating your own Android device or emulator for repetitive UI tasks and data extraction, and you are comfortable supplying your own LLM keys. Do not adopt it if you need to drive games, since the project states it has limited effectiveness there because games do not expose accessibility tree data, or if you need a mature, stable API.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap mobile-use targets: apps with no API
Most automation assumes an interface you can call. Mobile apps often have none. A banking app, a courier tracking screen, a field data collection tool: the only way in is the same one a person uses, tapping through the UI. mobile-use is built for that situation. It describes itself as an AI agent that controls an Android or iOS device using natural language, and the README frames the use cases as sending messages, navigating complex apps, and extracting data from any app into a structured format such as JSON. The intended audience is developers and QA engineers who already write Python and want to script a phone without maintaining brittle coordinate-based tests. The topics list on the repository includes qa and langgraph, which fits that reading. It is not a consumer product. You supply an LLM provider, an API key, and a device.
Planner, executor, and the accessibility tree as the sensor
The mechanism visible in the material is a graph of agent nodes. The README refers to "the agents that power mobile-use" in the plural, and the LLM configuration file is organised per agent node, which means you can assign a different model or provider to different stages of the run. The perception layer is the accessibility tree. The README's own caveat about games makes this explicit: games "don't provide accessibility tree data", and mobile-use is correspondingly weak on them. So the agent reads the structured UI hierarchy the operating system exposes, decides on an action, and issues it through the device control layer (ADB on Android, based on the connection troubleshooting text). Output is not fixed. You describe the shape you want in natural language, and the README gives the example of a JSON list of objects with sender and subject keys. That is the data flow: natural language instruction in, accessibility tree state read, actions dispatched, structured result out.
Getting it running with the Docker quickstart
The fastest documented path is the Docker quickstart, and the README notes it is currently Android only. You either plug in a physical Android device with USB debugging enabled through Developer Options, or start an emulator. Then, on Linux or macOS, you make the shell script executable and run it with a task and an output description. The README's example is `bash ./mobile-use.sh "Open Gmail, find first 3 unread emails, and list their sender and subject line" --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"`. On Windows the equivalent is `powershell.exe -ExecutionPolicy Bypass -File mobile-use.ps1` with the same two arguments. If you use your own phone, you have to accept the ADB connection prompts that appear on the device. The troubleshooting section adds a constraint worth reading twice: the script connects over IP, so the phone must be on the same Wi-Fi network as the computer. If it is not, you get the message about not being able to get a device IP.
Configuring models: .env, presets, and per-node overrides
Configuration has two layers. The first is `.env`, created by copying `.env.example`. That holds provider keys. The second is `llm-config.override.jsonc`, created by copying `llm-config.override.template.jsonc`, which lets you change models per agent node. Providers named in the README include OpenAI, Google, xAI, OpenRouter, MiniMax, and Anthropic. For MiniMax you set `MINIMAX_API_KEY` and either copy the `minimax` preset from `llm-config.defaults.jsonc` or set `"provider": "minimax"` on individual nodes; the documented models are `MiniMax-M2.7` and `MiniMax-M2.7-highspeed`, both with 200K context. For Anthropic you set `ANTHROPIC_API_KEY` and do the same with the `anthropic` preset, choosing between `claude-sonnet-4-6` and `claude-haiku-4-5-20251001`. Any OpenAI-compatible endpoint works: set `OPENAI_BASE_URL` and `OPENAI_API_KEY`, then point nodes at the `openai` provider. Google Vertex AI requires either ambient credentials via gcloud or workload identity, or `GOOGLE_APPLICATION_CREDENTIALS` pointing at a service account JSON file. The per-node override is the most interesting part of this design, because it lets you put a cheap model on routine nodes and a stronger one where judgement matters.
Where it breaks: games, Wi-Fi, and the emulator-only caveat on iOS
Three limitations are stated or strongly implied in the material. Games are the clearest. The README says mobile-use has "limited effectiveness with games as they don't provide accessibility tree data", which is a structural problem rather than a tuning problem: if the perception layer has nothing to read, no model swap fixes it. Canvas-rendered apps and anything using custom drawing will behave similarly, though the README only names games. Second, the Docker quickstart is Android only, and iOS support is mentioned in the feature list without a documented equivalent quickstart in this README, so iOS users are pointed at the local SDK quickstart instead. Third, the network requirement is a real operational constraint for CI: a device on a different subnet than the runner will fail before the agent starts. There is also a maturity signal in the README itself, which states the project is "quickly evolving" and invites bug reports. Treat the interface as moving. The release cadence supports that: v2.6.0 in October 2025, v2.9.0 in November 2025, v3.3.0 in January 2026.
How it differs from Appium-style UI testing
The obvious comparison is Appium or Espresso, the established mobile UI automation frameworks. They are deterministic. You write a selector, a test asserts on a result, and the same input produces the same output every run. mobile-use inverts that. You write an intent in natural language, an LLM decides which element to touch, and the path can vary between runs. That is a genuine trade-off, not a strict improvement. Appium gives you reproducibility and no inference cost; mobile-use gives you tolerance for UI drift and the ability to describe a goal instead of a script. For a regression suite that must pass or fail identically on every commit, Appium remains the right tool. For one-off extraction from an app you do not control, or for exploratory automation where writing selectors would take longer than describing the task, the LLM approach is the point. The README's data scraping feature, extracting from any app into a format you describe, has no clean equivalent in a selector-based framework.
Licence, maintenance, and what a version bump costs you
The licence is Apache-2.0, which permits commercial use, modification, and redistribution provided you keep the notices and state changes; it also includes a patent grant. That is a permissive choice, and it means you can embed mobile-use in a proprietary internal tool. This is not legal advice, and if you redistribute it you should read the LICENSE file rather than this summary. On maintenance cost, the material shows three releases in roughly three months and a README that describes the project as evolving quickly. Practically, that means two things to budget for: your `llm-config.override.jsonc` may need revisiting when defaults change, and the prompts or node structure may shift between minor versions. Because the model layer is pluggable, a provider deprecating a model is a config edit rather than a code change, which lowers one class of upgrade risk. The bigger unknown is behavioural drift: if the planner's prompting changes between versions, an automation that worked on v3.3.0 may need re-verification on the next release. Pin your version and test before upgrading.
Who should pick this up, and what to check first
mobile-use fits developers automating their own Android hardware for tasks that are tedious to script and stable enough to describe, and QA teams exploring intent-based tests rather than selector-based ones. The per-node LLM configuration is the feature most worth exploiting early, since it is the main lever on cost and quality. It does not fit anyone who needs to automate games, anyone who needs a frozen API surface, or anyone unwilling to manage LLM keys and per-token spend. The benchmark claim in the README, that the project was first to complete 100% of the AndroidWorld benchmark, is a claim made by the maintainers and points to their own benchmark page and an arXiv paper; verify it there rather than taking it at face value, since this review did not run the benchmark. The first thing to confirm on your own setup is the network path between your machine and the device, because that is the failure the README documents most concretely, and it happens before any model is called.
Editorial conclusion
Adopt mobile-use if you are automating your own Android device or emulator for repetitive UI tasks and data extraction, and you are comfortable supplying your own LLM keys. Do not adopt it if you need to drive games, since the project states it has limited effectiveness there because games do not expose accessibility tree data, or if you need a mature, stable API. Before committing, verify the Docker quickstart against your own hardware, confirm which LLM provider you will use by reading llm-config.defaults.jsonc, and check that your device sits on the same Wi-Fi network as your computer, which the troubleshooting section names as a hard requirement.
Community notes