test-prof: finding the slow specs in a Ruby test suite

TestProf profiles a suite you were going to run anyway

TestProf is a collection of different tools to analyze your test suite performance, and that narrow target shapes everything else about it. It does not run your tests. It does not replace RSpec or minitest, and it ships no assertion library of its own. What it does is sit beside a run you were going to execute anyway and report where the time went.

The audience is a Ruby developer whose feedback loop has gone slack. The README places testing inside a developer's feedback loop and, separately, inside a deployment cycle, then compresses the case into one line: slow tests waste your time making you less productive. A suite that finishes in seconds gives this toolbox nothing to report. A suite nobody can attribute to one spec file is exactly the gap it exists to close.

Keep the framing modest, though. Profiling never removes a slow test; it only makes one visible, and even that visibility costs a full run plus a report to read.

Plug'n'Play activation on top of ruby-prof and stackprof

The toolbox has two layers, and the first one is not its own. Under general Ruby profilers sit `ruby-prof` and `stackprof`. TestProf contributes a Plug'n'Play integration, meaning profiling switches on inside a test run rather than replacing the profiler you already know how to drive. Below that sit parts with no standalone equivalent: factories usage analyzers and profilers, ActiveSupport-backed profilers, RSpec and minitest helpers, and RuboCop cops.

The repository layout matches the split. `lib/` carries the library, `bin/` the executables, `config/` the defaults, `gemfiles/` the alternate dependency sets, `spec/` the project's own suite, and `docs/` its documentation. Three dotfiles carry the quality gate the project holds itself to: `.rubocop.yml`, `.rubocop-md.yml` and `forspell.dict`, whose names point at lint and spell checking over both code and prose. `.rspec` and `.jrubyrc` sit at the top level too, so RSpec options and a JRuby configuration live in the repository rather than in your shell.

One Gemfile entry, and nothing else to install

Installation is a single Gemfile entry and nothing else. The README gives this example:

group :test do
  gem "test-prof", "~> 1.0"
end

Placing the dependency inside `group :test` keeps it out of an ordinary application boot, which suits a tool whose entire job happens inside your tests. The installation section then ends on a joke, "And that's it)", and the joke is nearly accurate: no generator, no require line, no configuration file in the snippet.

What that snippet never settles is which profiler backs the ruby-prof and stackprof integrations. Neither gem is listed as a dependency of that block; both appear only as Plug'n'Play targets. The next honest stop is the runtime floor:

Ruby (MRI) >= 2.7.0
JRuby >= 9.3.0

Check those two numbers before anything else. A suite on an older interpreter sits out of range for the current line, and the version routing printed underneath decides which older line applies to it.

One install line leaves four Ruby generations behind

Copy that block into a project on Ruby 2.6 and the resolver hands you the newest build the constraint allows, because `~> 1.0` admits every 1.x version. The README's own note sends Ruby 2.5-2.6 users somewhere else: TestProf < 1.3, on separate routing lines that put Ruby 2.4 on <0.12.0, Ruby 2.3 on `~> 0.7.0` and Ruby 2.2 on < 0.7.0. Four interpreter generations sit behind that table, and the one-line installation does not speak to any of them.

The gap is not cosmetic. Ruby (MRI) >= 2.7.0 is the floor for the current line, so the 1.x build a modern resolver picks is not one the same table certifies for 2.5 or 2.6. Teams on those interpreters have to edit the constraint by hand, and anyone who copies the snippet verbatim inherits a resolution its own README warns against.

The RSpec side has a smaller version wall. RSpec features are supported at >= 3.5.0, with older RSpec versions pointed at TestProf < 0.8.0. The qualifier in parentheses is the interesting part: RSpec features only. The gem itself carries no RSpec floor, so a minitest project inherits a matrix it never needed.

The reported gains belong to other people's suites

The adoption list is the strongest pitch in the project and the least checkable part of it. Discourse reduced ~27% of their test suite time. Gitlab reduced 39% of their API tests time and improved factories usage. Both figures arrive as links to somebody else's ticket or post, and this repository publishes no benchmark of its own. Neither number says how long the suite ran beforehand, how many cores were available, or whether the work landed in CI or on a laptop.

There is a smaller signal in the same list. The named adopters, CodeTriage, Dev.to and Open Project among them, are followed by a link to an open issue where maintainers ask for more stories. An adopter list that ends by soliciting entries is partial by construction. Read those percentages as evidence that the toolbox moves suites in a useful direction, not as a number to budget against.

Factory analytics that never name the factory library

Factories usage analyzers and profilers are the most project-specific layer, and the README is thinnest exactly there. Nothing in it names the factory library those analyzers read, and no snippet shows how a profiling run is switched on: no RSpec tag, no environment variable, no rake task. The whole Usage section is one sentence, "Check out our docs", pointing at test-prof.evilmartians.io.

The repository ships a `docs/` directory beside the `lib/` and `spec/` trees and publishes that site separately, so the split is structural rather than an omission. It still puts a cost on the reader: the README answers what the toolbox contains, and the site answers how to invoke it. That is a documentation dependency, and it is the kind that ages quietly. The RuboCop cops and the RSpec and minitest helpers sit in the same category, named in the feature list as tools for writing faster tests, with no example and no configuration key attached.

Where a plain profiler already answers the question

Ask first whether you need a toolbox at all. If the question is which code inside one slow spec file burns the CPU, running `ruby-prof` or `stackprof` against that file yields a call graph directly, with no gem in your test group and no layer that has to understand factories or ActiveSupport. TestProf's contribution in that territory is Plug'n'Play activation inside a test run, which pays off when the unit of analysis is the whole suite.

Where the standalone profilers stop is where TestProf starts: factory usage counts, ActiveSupport-backed profilers, and helpers for RSpec and minitest that a sampling profiler has no concept of. A suite burning its time in factories needs a factory-aware counter, and that is the case the toolbox is built for.

The cost of finding out is small but not zero. MRI >= 2.7.0 and JRuby >= 9.3.0 are fixed floors, and RSpec features want >= 3.5.0. A project below those numbers cannot reach the factory analyzers through the current line at all.

Upgrade cost lands in the dependency matrix, not the code

Moving between versions costs you dependency resolution rather than code changes; no migration step appears in the README. The newest listed build is 1.6.0, dated 2026-03-18. Behind it sit 1.4.0 and 1.3.0, both dated 2024-08-12 and stamped minutes apart, which reads as a paired cut rather than two independent ones. Among the entries on record, nothing sits between 1.4.0 and 1.6.0, so that gap is wider than the dates alone convey.

The last push was on 2026-09-08 and the repository is not archived, so work is landing on master well after the 1.6.0 tag. The tree shows where that work goes: `.github/` behind the Build and JRuby Build badges, `gemfiles/` for version-specific dependency sets, `BACKERS.md`, and a `Rakefile` the project's own tasks run through. Evil Martians builds the gem, and adoption carries no licensing question, since it is open source under the terms of the MIT License.

Editorial conclusion

TestProf earns its place in a Rails-era Ruby suite where factory setup dominates the run and no one can say which spec is slow. It loses in a small suite, or on anything below Ruby 2.7.0, where a direct ruby-prof run answers the same question with nothing to install. Check three things before adopting: that your interpreter clears 2.7.0 and your RSpec clears 3.5.0, that the version your resolver picks is the one your Ruby generation is routed to, and that the factory analyzers at test-prof.evilmartians.io cover the factory library you actually use.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. test-prof/test-prof on GitHub