Phoronix Test Suite: Automated Benchmarking for Linux, BSD, macOS and Windows

What Phoronix Test Suite solves, and for whom

Running a benchmark by hand means downloading a source tree, resolving build dependencies, picking flags, and writing down the result somewhere. Repeat that across a dozen tests and two machines and the numbers stop being comparable. Phoronix Test Suite exists to remove that work: the README describes it as a platform that carries out tests in a fully automated manner from installation through execution and reporting, with the stated goal that tests be easily reproducible. The audience is engineers and reviewers who need the same workload executed the same way on more than one system, and who want the output stored rather than pasted into a spreadsheet. The framework is written in PHP and licensed under GPLv3. It is not a single benchmark; it is the machinery around a catalogue of them. That distinction matters when you evaluate it, because the quality of any individual number depends on the test profile you selected, not on the framework alone.

Test profiles, test suites and the OpenBenchmarking.org catalogue

The unit of work is a test profile: one test with a set of options, implemented as shell scripts plus XML files. A test suite is a single XML file that groups test profiles or other suites. The README states that more than 600 individual test profiles and more than 200 test suites are available by default through OpenBenchmarking.org, the collaborative storage platform developed alongside the suite. Results can be uploaded there as well, which is how the side-by-side comparison feature gets its data. The design is deliberately extensible: adding a benchmark means writing a profile, not patching the framework. That is the strongest argument for the architecture. The trade-off is that profile quality is uneven by nature, since each one is a separate set of scripts with its own build and runtime assumptions, and the README notes that not all profiles and suites are compatible with all CPU architectures even though the framework itself supports i686, x86_64, ARM, RISC-V and POWER 64-bit.

Installing Phoronix Test Suite on Ubuntu and other Linux systems

The only hard dependency is command-line PHP, version 5.3 or newer. A full PHP stack with a web server is not required. The README says the package is widely available from operating system package managers under names such as php, php7-cli, php5-cli, php7 or php8. After that, the suite can be installed system-wide or run locally from an extracted tar.gz or zip package without installation. The repository ships install-sh and install.bat for the two scripted paths. Once it is on the PATH, the first real use is a single command that downloads, builds, runs and reports a CPU test:

phoronix-test-suite benchmark smallpt

The README gives smallpt as its example of a simple CPU test profile. Expect the command to spend most of its time fetching and compiling the test before any measurement happens. To separate those phases, install and run are available individually:

phoronix-test-suite install smallpt
phoronix-test-suite run smallpt

To check what the framework detects before you benchmark anything, run system-info:

phoronix-test-suite system-info

That command prints the installed hardware and software the suite can see, which is the quickest way to confirm the machine is being read correctly.

Batch mode, regression testing and Phoromatic

Interactive prompts are the default, which is wrong for CI. Batch mode removes them: run batch-setup once to record the answers, then use batch-run instead of run. The README states there is also an integration path with git-bisect and other revision control systems for per-commit regression testing, plus system sensor monitoring. Phoromatic is the multi-machine layer: a web interface for driving Phoronix Test Suite clients on a timed, per-commit or other trigger-driven schedule, described in the README as designed for enterprise management of networked systems. This is where the project's scope becomes clear. It is not just a benchmark runner; it is a scheduling and collection system, and the value grows with the number of machines you point at it. A single workstation gains little from Phoromatic. A lab with ten identical machines under test gains a lot.

Where Phoronix Test Suite is the wrong tool

Portability is the first limit. The README is explicit that Linux is the most full-featured and well supported operating system for conducting tests, and that some non-basic functionality is not available under all platforms. If your target is Windows or macOS, treat the framework as usable but the coverage as partial, and verify the specific profile you need before planning around it. The second limit is maintenance. The last push to the repository was on 2022-07-03, and the most recent release listed is v10.8.4 from the same date. That is not a project you should expect to receive fixes for a broken profile or a new distribution's packaging quirks. The third limit is scope: this measures performance and pass/fail outcomes. It is not a correctness test framework, and the README frames profiles as representing performance benchmarks, unit tests and other quantitative or qualitative measurements, which means quality depends entirely on which profile you picked. If you need a small, self-contained benchmark you can read end to end in an afternoon, the framework is more machinery than the job requires.

How it compares with a single-purpose benchmark

The realistic alternative is not another benchmarking platform but a single benchmark you build and run yourself, such as a compiler benchmark or a ray tracer compiled from source with fixed flags. The difference is in what gets standardized. A hand-rolled benchmark standardizes nothing beyond your own script; you own the build, the flags and the result format. Phoronix Test Suite standardizes all three through the profile format and OpenBenchmarking.org, and it adds a result store you can compare against other people's submissions. The cost is a PHP runtime, a profile download step, and a dependency on a catalogue whose individual entries you did not write and cannot easily audit without reading shell scripts. For a one-off measurement on one machine, the hand-rolled benchmark is less work. For repeated measurements across machines or over time, the framework's standardization is the reason to accept the overhead.

Licence and the cost of staying current

The project is licensed under GNU GPLv3, and the README states it is developed by Phoronix Media in cooperation with partners. Running the suite internally does not create obligations beyond the licence terms, but redistributing a modified version does bring GPLv3 conditions into play; that is a general property of the licence, not legal advice, and your counsel should read the terms. On upgrade cost, the picture is simple and slightly uncomfortable: the last push was on 2022-07-03, so there is no steady stream of fixes to track. The practical cost is not upgrading, it is that new distribution releases, new compilers and new hardware may not be reflected in the profiles or in the framework's detection code. Budget time for reading and possibly patching profile scripts when a test fails to build on a current system, and check the ChangeLog and release-highlights.md in the repository before assuming a given behaviour changed in a version you are moving to.

Editorial conclusion

Adopt it if you need reproducible, scriptable benchmarks on Linux and are comfortable with a project whose last push was on 2022-07-03. Do not adopt it if you need a maintained release cadence or first-class non-Linux coverage, because the README itself notes some non-basic functionality is unavailable on other platforms. Before committing, verify that PHP CLI is installed on the target machines and run phoronix-test-suite system-info to confirm the suite can see the hardware you intend to measure.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes