# test-queue: a parallel test runner that keeps a history of how long your tests take

> test-queue is a Ruby gem that distributes tests across forked workers through a central queue, and it keeps build statistics between runs so that slower tests are handed out first. It bundles runners for testunit, minitest and rspec, supports distributed runs across several machines, and expects you to subclass its runner classes to isolate per-worker state.

**tmm1/test-queue** — parallel test runner for CI environments

- Repository: https://github.com/tmm1/test-queue
- Stars: 629 · Forks: 68
- Language: Ruby
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/tmm1-test-queue

## A central queue, and statistics carried between runs

test-queue describes itself as yet another parallel test runner, built using a centralized queue to ensure optimal distribution of tests between workers. The part that distinguishes it from the other options is stated plainly: it is specifically optimized for CI environments, because build statistics from each run are stored locally and used to sort the queue at the beginning of the next run. That single idea does the heavy lifting. A naive parallel runner hands out tests in file order, which means the slow integration test that happens to sit early in the suite gates every worker. test-queue reorders by how long things actually took last time, so the long pole starts first. The audience is a Ruby team with a suite that has grown slow enough that this matters, and the repository is a gem published on rubygems.org as test-queue, MIT licensed, with a Rakefile, an Appraisals file, gemfiles directory and separate spec and test directories.

## Bundled runners for testunit, minitest and rspec

The package ships three executables that can be used directly, testunit-queue, minitest-queue and rspec-queue, and the README shows how they are invoked:

```console
$ minitest-queue $(find test/ -name \*_test.rb)
$ rspec-queue --format progress spec
```

The bundled binaries are convenient, but the README is clear that the underlying classes, TestQueue::Runner::TestUnit, TestQueue::Runner::Minitest and TestQueue::Runner::RSpec, are built to be subclassed by your application, and it recommends checking a new executable into your project using one of these superclasses. That is the intended integration path, and it is how you get at the hooks that make parallel runs safe. The recommended shape of that file is small:

```console
$ vim script/test-queue
$ chmod +x script/test-queue
$ git add script/test-queue
```

The last step matters more than it looks. A project that runs its suite in parallel needs its own runner in version control, because that file is where the isolation logic lives and where it gets reviewed.

## fork(2) means you own the isolation problem

The design section states that test-queue uses a simple master plus pre-fork worker model, where the master exposes a Unix domain socket server which workers use to grab tests off the queue. Since it uses fork(2) to spawn off workers, the README is emphatic that you must ensure each worker runs in an isolated environment, and the mechanism for that is the after_fork hook with a custom runner, used to reset any global state. The example runner in the README shows how much of this is your job: it points each worker at a separate MySQL database by appending the worker number to the database name, reconnects Redis to a per-worker database index, creates the per-worker databases in a prepare method, and hooks an around_filter to time each suite. The example also shows the remote master announcing its output location to the central master. The design is reasonable, and the cost is that isolation is not optional and not automatic.

## Environment variables are the whole configuration surface

The README documents the configuration entirely through environment variables, and the list is where you should look before choosing the tool. TEST_QUEUE_WORKERS sets the number of workers per master, defaulting to all available cores. TEST_QUEUE_VERBOSE shows results as they become available and defaults to 0. TEST_QUEUE_SOCKET sets the Unix socket path, or a TCP address and port pair, used for communication, with a default of a path under /tmp. TEST_QUEUE_STATS is the path to the cache of build statistics, defaulting to .test_queue_stats, which is the file that makes the reordering work across runs. TEST_QUEUE_FORCE takes a comma separated list of suites to run. TEST_QUEUE_SPLIT_GROUPS splits tests by example rather than by example group, which the README says is faster for tests with short setup time such as selenium, and it is RSpec only, with a :no_split tag for example groups you want left alone. The remaining variables, TEST_QUEUE_RELAY, TEST_QUEUE_RELAY_TIMEOUT, TEST_QUEUE_RELAY_TOKEN and TEST_QUEUE_REMOTE_MASTER_MESSAGE, all belong to the distributed mode.

## Distributed mode needs a matching token on every master

The second half of the design is a distributed mode where additional masters share the workload and relay results back to a central master. The README is specific about the requirement: the central master must listen on a TCP port, additional masters are booted in relay mode to connect to the central master, and remote masters must provide a TEST_QUEUE_RELAY_TOKEN matching the central master's. The three commands it gives show the three roles:

```console
$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_SOCKET=0.0.0.0:12345 bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_RELAY=0.0.0.0:12345  bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 ./test-multi.sh
```

The token is the whole authentication story, a shared string that must match on both sides, and the README points at the Jenkins Parameterized Trigger Plugin as a simple way to wire this up. The design is straightforward and old in the good way, since the socket model has not changed in years. The trade-off is that a distributed run is a distributed system with a shared secret and a TCP port, so the operational surface is larger than the local case, and the project does not document what happens when a remote master dies mid-run.

## What to reach for instead, and what the project points at

The See also section names two alternatives, and the difference in approach is instructive. Shopify's rails_parallel and grosser's parallel_tests both take the simpler route: they shard the test files across a fixed number of processes, statically, without a queue and without any history of how long anything took. That approach is easier to reason about, needs no socket and no per-worker setup beyond the standard database sharding, and for a suite of moderate size it is often entirely sufficient. test-queue is for the case where that is not enough, where a handful of slow files dominate the wall clock and you want the ordering to adapt. The cost of that adaptivity is the machinery: a master process, a socket, forked workers, a statistics cache that has to be written somewhere and might be stale, and the requirement that your application subclass the runner so that each worker is isolated. The other gap in the project worth noting is that the README does not document what the runner does when a test leaves a stray process or an open database connection behind, which is precisely the kind of failure that only shows up on a busy CI machine.

## Maintenance: a 0.x line with a 2023 gap and a 2026 release

The release history is worth a second look, because it is not a smooth curve. v0.11.0 was published on 2023-07-01, v0.11.1 on 2023-12-20, and then there is a long gap before v0.12.0 on 2026-01-10. The last push to the repository was on 2026-06-22. So the project is alive, but the gap between the two most recent tags is more than two years, and the version stays below 1.0, which in Ruby convention signals exactly what it usually signals, that the API may still move. The repository has continuous integration through GitHub Actions, a rubocop configuration, and both a spec directory and a test directory, which suggests the gem is tested with its own framework and with the runners it provides. The licence is MIT. For an adopter, the practical point is that you are depending on a gem maintained by a single author with an irregular release cadence, so pinning a version and reading the CHANGELOG for the 0.12.0 jump is worth the five minutes it costs.

## Conclusion

test-queue fits a Ruby team with a slow suite that wants to reorder tests by past duration rather than by file order, and that can afford the discipline of isolating each worker through an after_fork hook. It does not fit a suite that is already fast, since the overhead of a master, a socket and a fork per worker buys nothing there, and it does not fit a project that cannot isolate shared state. Before adopting it, subclass the runner for your framework, reset global state in after_fork, and confirm the CI environment can give each worker its own database, because the example code shows exactly how much of that work is on you.

## FAQ

### How do I run my tests in parallel with test-queue?

Use the bundled binaries directly, for example minitest-queue with your test file list or rspec-queue with a spec directory, or subclass TestQueue::Runner::Minitest, TestQueue::Runner::TestUnit or TestQueue::Runner::RSpec and check an executable into your project that uses it.

### What does test-queue do differently from parallel_tests?

test-queue uses a central queue and stores build statistics from each run locally, then uses those statistics to sort the queue at the start of the next run, so slow tests are handed out first. The alternatives named in the README, rails_parallel and parallel_tests, shard test files statically instead.

### How do I keep workers from interfering with each other?

Because test-queue forks workers, you must ensure each worker runs in an isolated environment, and the documented mechanism is the after_fork hook in a custom runner subclass. The README's example resets per-worker MySQL and Redis state in that hook.

### How do I set up a distributed test-queue run?

The central master must listen on a TCP port, additional masters boot in relay mode to connect to it, and every remote master must set a TEST_QUEUE_RELAY_TOKEN matching the central master's. The README also points at the Jenkins Parameterized Trigger Plugin as a simple way to wire this up.

## Sources

- [Issues](https://github.com/tmm1/test-queue/issues)
- [License: MIT](https://github.com/tmm1/test-queue/blob/master/LICENSE)
- [README](https://github.com/tmm1/test-queue/blob/master/README.md)
- [Releases](https://github.com/tmm1/test-queue/releases)
- [tmm1/test-queue on GitHub](https://github.com/tmm1/test-queue)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/tmm1-test-queue