# Marathon schedules mobile tests around flakiness instead of just running them in parallel > Marathon is a cross-platform test runner for Android and iOS that treats test duration and pass probability as the two inputs to scheduling. It batches, shards, sorts and retries tests, manages device pools, and ships a Marathonfile plus a Marathondevices file, and the repository is GPL 2.0 with a separately licensed parser component. **MarathonLabs/marathon** — Cross-platform test runner - Repository: https://github.com/MarathonLabs/marathon - Website: https://marathonlabs.github.io/marathon/ - Stars: 646 · Forks: 128 - Language: Kotlin - License: GPL-2.0 - Published: 2026-10-21 · Updated: 2026-10-21 · Language: en - Canonical page: https://hysenlabs.com/projects/marathonlabs-marathon ## Two inputs drive the schedule: expected duration and pass probability Marathon is a project that helps execute tests in the shortest time possible, and the README splits that goal into two parts. The first is stability of test execution, adjusting for flakiness in the environment and in the tests. The second is performance, achieved through high parallelization. The framing that makes it different from a plain parallel runner is this. A test run can only finish as quickly as possible if the execution is planned with regard to the expected duration of each test, and the flakiness problem is addressed by tracking the probability of a test passing. Those two quantities are the inputs to the scheduler. Three mechanisms follow from that. The flakiness strategy queues up preventive retries for tests which are expected to fail during the run according to the current real-time statistical data, so a test that is known to be shaky gets a retry budget before anyone looks at the results. The sorting strategy forces long tests to be executed first, so that if an unexpected retry does occur it lands at the end of the run rather than in the middle of it. And when everything else fails, the runner reverts to post-factum retries, but tries to limit their impact with retry quotas. That third point is the difference between a runner that hides flakiness and one that bounds it. A retry quota means the number of retries is capped, so a permanently broken test fails the build instead of consuming the whole run. The project also implements the surrounding concepts: test batching, device pools, test sharding, test sorting, preventive retries and post-factum retries. The README says most of these default to conservative values, and that custom configurations are encouraged for anyone who wants to optimise performance or stability. ## The Marathonfile holds the vendor specifics, and there are two of them Configuration lives in a file called Marathonfile in the root of the project, and the vendor section describes the platform specific details. For Android it names the application APK and the test application APK: ```yaml name: "My application" outputDir: "build/reports/marathon" vendorConfiguration: type: "Android" applicationApk: "dist/app-debug.apk" testApplicationApk: "dist/app-debug-androidTest.apk" ``` For iOS the same shape points at a bundle and a UI test bundle: ```yaml name: "My application" outputDir: "derived-data/Marathon" vendorConfiguration: type: "iOS" bundle: application: "sample.app" testApplication: "sampleUITests.xctest" testType: xcuitest ``` The type key selects the platform implementation, and the testType of xcuitest selects the UI testing driver. Output paths differ per platform too, since Android writes reports under the Gradle build directory and iOS uses the derived data directory. The repository ships ready-made examples under sample/: a Marathonfile, android-app, android-cucumber-app, android-library and ios-app. The presence of a Cucumber sample is notable, because it means the runner is not limited to the standard instrumentation or XCUITest entry points. The top level entries also show the shape of the codebase: cli/, configuration/, core/, vendor/, report/, html-report/, analytics/ and buildSrc/, alongside the Gradle Kotlin DSL build files. The analytics directory is a detail worth noting for anyone with data governance questions, since the project is explicitly about gathering statistics on test behaviour and a module named analytics implies the runner collects its own data. ## iOS needs an explicit device list because it has no remote discovery Android devices are discovered for you. iOS simulators are not, which is why the README requires a Marathondevices file listing the macOS instances and simulators to use. The reason is stated plainly: since iOS doesn't have any way to discover remote execution devices you have to provide your simulators. The file describes workers, each with a transport and a set of devices: ```yaml workers: - transport: type: local devices: - type: simulator udid: "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" - type: simulatorProfile deviceType: com.apple.CoreSimulator.SimDeviceType.iPhone-13-mini ``` Two device types appear. A simulator is identified by its UDID, which is a specific instance, while a simulatorProfile names a device type such as the iPhone 13 mini, which lets Marathon provision or match a simulator of that profile. The README says Marathon can use pre-provisioned simulators but can also provision new ones if needed. The transport of type local means the worker is the machine where Marathon itself runs. The same file supports remote macOS instances over SSH, and the README notes that the instance where you run Marathon is not limited to macOS, so with remote macOS workers you can start a run from Linux. That is what makes the device pool a real pool rather than a local simulator list, and it is also why the hardware farm story in the project description holds: custom farms plug in through the same worker abstraction. More detail on providing devices lives in the workers section of the documentation. ## Running it is one command from the project root The execution step is deliberately undramatic. You start the runner in the root of your project: ```bash $ marathon XXX [main] INFO com.malinskiy.marathon.cli.ApplicationView - Starting marathon XXX [main] INFO com.malinskiy.marathon.cli.config.ConfigFactory - Checking Marathonfile config ... ``` The log lines come from a Kotlin logging setup, with the logger names com.malinskiy.marathon.cli.ApplicationView and com.malinskiy.marathon.cli.config.ConfigFactory, and the first two entries show the startup path and the config check. The runner finds the Marathonfile itself, so there is no path argument to pass in the simple case. The requirement is Java Runtime Environment 17 or higher. That is the only stated runtime requirement, and the primary language of the project is Kotlin with Gradle Kotlin DSL build files, a gradlew wrapper and a kotlin-version file at the root. The project is developed on a develop branch rather than master, which is a small but real signal: the default branch named develop usually means the released artifacts come from a separate release line, and the version badges and release links point to tagged releases rather than to the branch tip. ## The licence is the one thing to read before you wire it into a build The Marathon codebase is GPL 2.0, and the README spells out an exception structure that is easy to skim past. The codebase is under the GPL 2.0 LICENSE, with following optional components under specific licences, and the one named component is libxctest-parser with its own licence reference. For an open source project that is a normal arrangement, since the parser is a separately licensed component rather than part of the GPL corpus. For a commercial team, GPL 2.0 on the runner is the fact that matters: it carries copyleft obligations, and whether those reach your own build depends on how you link to and distribute the runner. That is a question for your own legal review rather than something this article can settle. The repository also carries a CODEOWNERS file and an ADOPTERS.md, and the contributing documentation is linked from the README. The default-detekt-config.yml at the root is the static analysis configuration for Kotlin, so the code quality tooling is part of the repository rather than external. Around the core project there is a commercial counterpart. Marathon Cloud is described as a scalable testing-as-a-service product built by the same creators, with cloud testing infrastructure for running tests in less than 15 minutes, positioned for startups and large enterprises alike. The open source runner is the part you can read and self host, and the cloud product is the managed form of the same idea. ## Where the runner stops and your own infrastructure starts The honest limits are visible in the configuration model. The scheduling intelligence depends on statistical data about past runs, and the preventive retry strategy is described as working according to the current real-time statistical data. A first run on a new project has no such data, which means the flakiness strategy has nothing to act on and you get the conservative defaults instead. The benefit accumulates over time rather than appearing on day one. The second limit is device capacity. The sorting strategy only pays off if there is a device pool wide enough to run the long tests concurrently, and for iOS that pool has to be described by hand because the platform offers no remote discovery. A team with one Mac and four simulators will see the batching and sharding work and the sorting work largely idle, because the run is bounded by hardware rather than by the scheduler. The third limit is platform coverage. The project ships Android and iOS vendor implementations, and the documentation also refers to an API for use with custom hardware farms and more techstacks, but any other platform means writing an adapter, which is what the ADOPTERS.md file is for. Compared with a stock runner such as the one your platform already provides, the difference in approach is scheduling rather than execution. A stock runner decides which tests to run and in what order within one device. Marathon decides which tests to run, on which device, in what order, and whether to retry, using duration and pass probability as its inputs. That is a different layer of the problem, and it only pays when the suite is big enough for those decisions to matter. ## Conclusion Adopt Marathon if your mobile test suite is slow or flaky enough that parallel execution alone is not enough, since the scheduling model explicitly uses expected duration and pass probability to order the run. Do not adopt it for a small suite, because the configuration surface of a Marathonfile, a Marathondevices file and device pool management is real work for a handful of tests. Verify first that your Java runtime is 17 or higher, that the vendor section matches the platform you run, and that you have read the GPL 2.0 terms, since the libxctest-parser component is under its own separate licence. ## FAQ ### What is a Marathonfile and what goes in it? A Marathonfile in the project root holds the name, the output directory and a vendorConfiguration section. For Android it names the applicationApk and testApplicationApk, and for iOS it names the application bundle, the test bundle and a testType of xcuitest. ### Why do I need a Marathondevices file for iOS? Because iOS offers no way to discover remote execution devices, so you have to provide your simulators. The Marathondevices file lists workers, each with a transport and devices, using a udid for a specific simulator or a deviceType for a simulator profile, and remote macOS instances can be connected over SSH. ### How does Marathon handle flaky tests? The flakiness strategy queues preventive retries for tests expected to fail during the run according to current real-time statistical data, the sorting strategy runs long tests first so an unexpected retry lands at the end, and post-factum retries are used as a fallback with retry quotas limiting their impact. ### What are the requirements for running Marathon? Marathon requires a Java Runtime Environment of 17 or higher. Configuration is a Marathonfile in the project root, and for iOS you also need a Marathondevices file listing simulators or remote macOS workers. ### What licence does Marathon use? The Marathon codebase is GPL 2.0, with optional components under specific licences, and libxctest-parser is named as one of them with its own licence. The repository also contains a LICENSE file, and Marathon Cloud is offered separately as a commercial testing-as-a-service product. ## Sources - [License: GPL-2.0](https://github.com/MarathonLabs/marathon/blob/develop/LICENSE) - [MarathonLabs/marathon on GitHub](https://github.com/MarathonLabs/marathon) - [Project website](https://marathonlabs.github.io/marathon/) - [README](https://github.com/MarathonLabs/marathon/blob/develop/README.md) - [Releases](https://github.com/MarathonLabs/marathon/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/marathonlabs-marathon