Airtest: image-recognition UI automation that needs no access to the app under test

Why matching pixels beats reading a UI tree here

The README leads with a single sentence that explains the whole design: Airtest uses image recognition technology to locate UI elements so that you can automate games and apps without injecting any code. That last clause is the differentiator. Espresso, XCUITest and Appium all reach into the application under test through a debug bridge or an accessibility interface, which gives precise element handles and rich assertions in exchange for requiring you to own and instrument the app.

Games generally cannot be instrumented that way. A Unity title compiled into an APK gives you no test IDs, no view hierarchy and no debug hook you are permitted to add, and if you could add one it would no longer be the build your players run. Airtest's answer is to keep a PNG of the thing you want to tap, find it on screen, and tap where it found it. The consequences follow from that.

Locating an element becomes a template match with a confidence threshold rather than a query, so the API surface is `touch(Template(...))` instead of a selector. Assertions become `assert_exists(Template(...))`, which checks that a picture is on screen, not that a field equals a value. Failures become waits, because the framework retries until the picture appears or a timeout expires. That retry loop is the single most important thing to understand about how these tests behave in a flaky environment: they hide intermittent failures by converting them into slow tests rather than fast red ones.

The API in eleven lines, and what it tells you about the model

The README's example is the shortest useful summary of the framework, and it is worth reading line by line because every call is one of the four categories the README names: connect, install, drive the interface, assert.

from airtest.core.api import *

# connect an android phone with adb
init_device("Android")

install("path/to/your/apk")
start_app("package_name_of_your_apk")
touch(Template("image_of_a_button.png"))
swipe(Template("slide_start.png"), Template("slide_end.png"))
assert_exists(Template("success.png"))
keyevent("BACK")
home()
uninstall("package_name_of_your_apk")

The commented alternative line in the original, `connect_device("Android:///")`, shows the connection string form, which is the same URI shape you pass on the command line later.

Two things stand out. First, the names are flat and device agnostic: `install`, `start_app`, `touch`, `swipe`, `keyevent` and `home` do not mention Android, iOS or Windows anywhere. That is the cross platform promise being kept at the API level rather than in the marketing copy. Second, `Template(...)` takes an image file, not an element or a selector. Every one of those calls is doing a picture lookup against the current framebuffer.

The README also flags something in bold and caps: do not forget to make assertions of the expected result. That is advice about a real failure mode, since a script that taps and swipes with no assertions passes whether or not the app did the right thing.

AirtestIDE, .air directories, and why the CLI exists

Airtest has two front ends and they produce the same artefact. AirtestIDE is a GUI tool for recording and running cases, and the README describes its workflow as create, run, report. What it writes is a `.air` directory, not a script, and that directory is the unit of execution everywhere else.

The CLI exists because AirtestIDE is a desktop application and device farms are not. Running `.air` cases from the command line lets a host machine execute a case built on someone's laptop:

airtest run "path to your .air dir" --device Android:///
airtest run "path to your .air dir" --device "Windows:///?title_re=Unity.*"

That second line is the interesting one. The device URI for Windows matches on window title with a regular expression, which is how you target one window among many on a desktop machine. The same URI grammar handles the Android case with `Android:///`, so a device farm scheduler can pass the target in as a single string.

Reporting is a separate subcommand, which makes sense when a farm wants to aggregate results after the run rather than during it:

airtest report "path to your .air dir"

The README notes the same functionality is available as a Python module via `python -m airtest run`, so a CI job that cannot put `airtest` on the PATH can still run the suite. A worked example lives at `playground/test_blackjack.air`, and a separate repository, `multi-device-runner`, is linked for fan-out scheduling. The README also states that HTML reports with detailed information and screen recording are what let you locate a failure point after the fact, which matters more than usual when the assertion is a picture.

Poco for when the widget hierarchy is actually available

Image matching is the general case, not always the right one. Poco is a separate project from the same organisation, and the README describes it as adding the ability to directly access the object hierarchy across the major platforms and game engines, allowing instructions to be written in Python for more advanced automation.

That is the natural counterpart to Airtest rather than a replacement for it. A game built on a supported engine typically exposes its own scene graph, so a widget query returns an actual button with an actual label, and you can assert on text rather than on the presence of a picture. Airtest can still supply the things a UI object model does not: device connection, app install and launch, screen capture, and the swipe and gesture vocabulary.

The split is worth stating explicitly when you choose a tool, because the failure modes differ in opposite directions. Airtest breaks when pixels change, so a new UI build invalidates templates and Poco breaks when the engine does not expose a hierarchy, so an unsupported engine leaves you with nothing. Projects that support both usually keep image tests for end to end flows and object queries for detailed assertions.

The framework also has a namesake ecosystem project worth knowing about. The README credits the stf mobile device farm, NetEase's own ATX agent tooling, and pywinauto, which is what the Windows implementation is built on. That credit list is effectively a dependency map for the platform layer.

Dependencies and the packaging decisions in setup.py

`requirements.txt` is the practical answer to what a machine needs before a test can run, and the pins are as informative as the names:

Jinja2>=2.8
Pillow>=3.4.0
mss==6.1.0
numpy<2.0
opencv-contrib-python>=4.4.0.46, <=4.6.0.66
facebook-wda==1.5.0
pywinauto==0.6.3

Pillow and OpenCV are the image stack doing template matching. `mss` is a screen capture library, pinned exactly, which matters because screen grabbing behaves differently across platforms and a floating version is how test machines start producing black frames. `facebook-wda` is the WebDriverAgent client for iOS. `pywinauto` at a fixed 0.6.3 is the Windows automation layer. `numpy<2.0` and the upper bound on OpenCV both point at a codebase that predates the NumPy 2 migration, which matters for anyone building a fresh environment today.

`setup.py` does something more interesting than declare metadata. It has an `is_docker()` function that inspects `/proc/self/cgroup` for the string `docker`, and when it returns true it swaps the OpenCV dependency for the headless build pinned at 4.5.5.64. That is a real operational detail rather than trivia: a container has no display server, and the full OpenCV wheel will fail to import there. The same function also handles the Windows case by appending `pywin32`, and pins older versions of `facebook-wda` for Python 3.7 and below. The version string is read out of `airtest/utils/version.py` rather than written in two places.

Maintenance pace, licensing and what is missing from the README

Two facts about the project's current state are worth stating before anyone invests time in it. The last push was on 2026-03-23, and the newest release tag is v1.4.3 published on 2025-12-04. That is a gap of several months between the newest tag and the most recent commit, which is a normal pattern for a project with a slow release cadence rather than a sign of abandonment, but it does mean trunk and the installed package differ.

The releases themselves are terse and written in Chinese. v1.4.3 pins a `facebook-wda` version. v1.4.2, published the day before on 2025-12-03, adds support for devices on iOS 17 and above and introduces a `screen_size` interface on iOS to replace the previous `window_size`, which is the kind of change that breaks existing cases on newer hardware. v1.3.6 from 2025-08-12 has an empty body. Read these as a versioned changelog with three entries rather than as documentation.

Licensing is Apache-2.0, which is permissive and unusual for a tool built in a game company's internal environment. Continuous integration still points at Travis CI through the badge in the README, while the repository contains a `.travis.yml` and a `github/` directory, so the migration to GitHub Actions may be partial. There is also a `benchmark/` directory at the repository root, which is unusual for a test framework and worth a look if you care about the cost of a template search.

The install section has one genuinely platform-specific step that trips people up. On macOS and Linux the bundled adb binary needs execute permission:

chmod +x adb

The README shows this being run inside the installed package at `airtest/core/android/static/adb/mac`, with a linux variant beside it. A pip install alone leaves you with a non-executable adb on those platforms, which fails at connect time with an error that does not obviously point at file permissions.

Editorial conclusion

Airtest solves a specific problem well: automating an interface you cannot instrument. Games rendered with Unity, packaged desktop applications and anything behind a canvas have no accessibility tree to query, so an image match against a stored template is not a fallback here, it is the mechanism. The cost is exactly what that mechanism implies. A resolution change, a theme switch or a localised string breaks a template, and the failure surfaces as a timeout rather than a useful diff. Pair it with Poco when the engine does expose a widget hierarchy, since Poco is designed to do for UI objects what Airtest does for pixels. Start with `pip install -U airtest`, write one case in the README's style, and check the report output before building a suite on it. The last push was on 2026-03-23 and the newest release tag is v1.4.3 from 2025-12-04, so plan around the version you install rather than around trunk.

Frequently asked questions

Official sources

1. AirtestProject/Airtest on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases