Caliper: the Java microbenchmarking library Google tells you to replace

The README tells you not to use it

The most direct thing in this repository is a section titled Prefer JMH or Jetpack Microbenchmark, and it says that for new benchmarks you should use a tool other than Caliper.

The reasoning given is accuracy for the JVM case, with JMH described as generally providing more accurate results than Caliper and a link to a discussion on that claim. For Android the recommendation is the Jetpack Microbenchmark library, which is the platform's own benchmarking solution rather than something maintained by Google as a general library.

So the position of Caliper today is odd but clear. The library is not deleted, the repository is not archived, and the last push was on 2026-10-14, so the code still moves. Yet the tool is described as deprecated in the project description and in the README's own guidance. That combination means Caliper is in maintenance mode: it still builds, still receives fixes, but it will not gain features and it will not be where new measurement infrastructure comes from.

The other piece of history is in the notice at the top. The Caliper web application that was at microbenchmarks.appspot.com was shut down on 2022-08-15, which removed the hosted service where benchmark results used to be published and shared.

A runner and a worker split

The repository layout reveals the architecture, and it is the part worth understanding if you are maintaining an existing Caliper setup.

There is a caliper-runner module and a family of worker modules, caliper-worker, caliper-worker-jvm and caliper-worker-android. The split is the design: the runner is the part that orchestrates a benchmark run and produces the results, while the worker is the part that executes the benchmark code in the target environment. Having separate JVM and Android workers is what lets the same runner drive a benchmark on a desktop JVM and on a device.

Around those sit caliper-api, the public API surface, caliper-core, caliper-util for shared utilities, caliper-examples, and the top level caliper module that ties them together as the artifact the build command installs.

The build is a Maven multi-module project, with the parent pom.xml at the root. The example module is worth noting for a different reason: caliper-examples is where a new user looks to see how a benchmark is written and what a Caliper benchmark looks like in practice, and those examples are effectively the documentation for people who cannot follow the migration advice to JMH.

Building the JVM version

The README is explicit that the JVM version is the only supported version at the moment, and that building it takes one Maven command:

mvn clean install -am -pl caliper

The flags carry most of the meaning. -am builds the required upstream modules in the reactor as well, and -pl caliper restricts the build to the module named caliper, so the other modules are built only as dependencies rather than as the goal. The result is the artifact that the Maven coordinates in the build file refer to.

Because the repository is a multi-module Maven build with a parent pom, this command also works from a clone where you intend to consume the library locally, for example during an experiment on an existing benchmark suite. Nothing in the build description suggests a need for a special toolchain beyond a JDK and Maven, and the absence of any mention of a test suite command in the README suggests the repository's own verification is part of the standard Maven lifecycle rather than a separate documented step.

The Android artifacts are in a transitional state

Caliper has a number of Android related artifacts, and the README does not soften the situation. They are described as being in a transitional state, with no easy way to run Caliper benchmarks on Android currently available.

If you still need to build them, the command is the module variant of the JVM build:

mvn clean install -am -pl caliper-android

and the build has an environment requirement that is easy to miss. ANDROID_HOME must be set to the location of an Android SDK that contains the file platforms/android-25/android.jar. The README offers two alternatives to that variable: passing -Dandroid.home with the SDK directory to the mvn command, and passing -Dandroid.sdk.version to pick a version other than 25, with a warning that the build may not work with a version lower than 25.

This is the clearest case in the repository of a legacy dependency that constrains your environment. A build that insists on an SDK layout from several Android generations ago is not something you want to add to a new project, and the combination of transitional status, no supported run path and a pinned platform level explains why the project points Android users at the Jetpack Microbenchmark library instead.

Licensing and the state of the code

The licence is Apache-2.0, with the text in the COPYING file at the repository root alongside the README, the pom.xml and the module directories. There is no separate licence file per module, which is what you would expect from a project distributed as a single Maven artifact set rather than as independently licensed components.

The release record is thin. The only tagged release listed is v1.0-beta-3, dated 2021-07-14, and the version string carries a beta marker. There are no recent releases even though the repository received a push on 2026-10-14, which tells you that the maintenance activity in the last few years has been fixes and dependency work rather than new tagged versions.

For a team deciding whether to stay on Caliper, that combination is the crux. The code is present, the build works, and the project is not frozen. But there is no stream of releases to upgrade to, no changelog to plan a migration against, and the benchmark measurement semantics of a beta-era tool are not the ones the project would point a new user toward.

Where Caliper is the wrong tool

The README answers the adoption question directly, and it is worth restating as a decision rather than a recommendation.

If you are writing new JVM benchmarks, JMH is the tool the project points to, on the grounds that it produces more accurate results. If you are writing Android benchmarks, the platform's own Jetpack Microbenchmark library is the route. Either way, starting a new suite in Caliper means adopting a tool whose own maintainers would rather you did not, and the accuracy argument means the numbers you produce would be compared unfavourably with what a colleague gets from JMH.

The second wrong-tool case is Android in general, and it is not subtle. There is no easy way to run Caliper benchmarks on Android currently, the artifacts are transitional, and the build pins an old platform level through the android.jar requirement. That is three separate reasons the Android path is a dead end for anything new.

The third case is a project that has outgrown its need. If the only reason Caliper is in your build is that it was there a few years ago, the work is to port the suite, and the cost of that port is the real budget line, not the effort of adding JMH to the pom. The reverse case is legitimate too: an existing multi-module project with a large Caliper suite and no scheduled migration can keep running the tool as it is, since the library still builds and still gets fixes.

The final point is about results. The hosted application where Caliper benchmark results were published and compared was shut down on 2022-08-15, so any workflow that assumed a hosted place to store or share those results needs somewhere else to put them.

Editorial conclusion

Adopt Caliper only to keep existing benchmarks running, for example when a Maven multi-module project already depends on caliper-core and rewriting the suite in JMH is not on the roadmap. Do not start new benchmark work with it, because the project itself directs you to JMH for JVM code and to Jetpack Microbenchmark for Android, and the hosted Caliper web application that once hosted results was shut down on 2022-08-15. Verify three things first: whether your benchmarks depend on the Android worker modules, since the Android artifacts are described as transitional with no easy way to run them; that your build satisfies the Android requirements if you touch those modules, including ANDROID_HOME pointing at an SDK with platforms/android-25/android.jar; and how long you will carry an existing Caliper suite, since the only tagged release is v1.0-beta-3 from 2021-07-14 while the repository itself still receives commits, with the last push on 2026-10-14. The licence is Apache-2.0.

Frequently asked questions

Official sources

1. google/caliper on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases