Mutant for Ruby: mutation testing, and the paperwork it does not add up

What it actually does, in one diff

Mutation testing is easy to describe badly. The honest version: take a method that is already covered, rewrite one operator or one literal, run the tests again, and record whether anything failed. If nothing failed, the test suite would have accepted behaviour nobody asked for, and that is the mutation that survives.

The README demonstrates this with the smallest possible example. The subject is a value object:

# lib/person.rb
class Person
  def initialize(age:)
    @age = age
  end

  def adult?
    @age >= 18
  end
end

And the tests that supposedly cover it:

# spec/person_spec.rb
RSpec.describe Person do
  describe '#adult?' do
    it 'returns true for age 19' do
      expect(Person.new(age: 19).adult?).to be(true)
    end

    it 'returns false for age 17' do
      expect(Person.new(age: 17).adult?).to be(false)
    end
  end
end

Both tests pass. The suite looks responsible: it exercises a true case and a false case, the classic pairing people write when asked for coverage. Run mutant and the boundary falls out:

gem install mutant-rspec
mutant run --use rspec --usage opensource --require ./lib/person 'Person#adult?'

 def adult?
-  @age >= 18
  @age > 18
 end

Nobody tested `age == 18`. The pair of tests straddles the boundary without landing on it, which is exactly the shape of gap that line coverage reports as 100 percent. Note also that the two arguments chosen are 19 and 17 rather than 18 and 17, which is the detail a coverage-first mindset produces: enough spread to look thorough, no landing on the decision boundary.

An alive mutation is a decision, not a defect

The framing the project insists on is worth more than the operator list, because it changes what a failure means. A surviving mutation is not automatically a bug in your code and not automatically a missing test. It is a fork with exactly two branches: the mutated code is the correct behaviour and the original was redundant, so accept the simplification; or the original code was correct and nothing verifies that particular behaviour, so write the test.

That is why the operator catalogue is wide rather than clever. The README lists arithmetic, logical and bitwise operators, statement removal, return value modification, and v0.16.0 added integer overflow boundary mutations, which is the category that catches the boundary case above in general. The 0.15.1 release added alive mutation explanations directly into the CLI report, with the stated goal of improving how agents read a mutant report.

That goal deserves scrutiny rather than acceptance. A tool that prints instructions aimed at machine readers is a tool whose report format is becoming an interface contract. If you build anything around mutant output, pin the version and expect the report text to change; v0.16.2, the current release, is a one-line fix for a crash on non-UTF-8 or binary log output, which is the kind of change that reshapes what a parser can assume.

The nomenclature docs, the configuration docs and the per-framework integration docs are the parts worth your time. Operators are AST-level, there is a dedicated document on AST pattern matching for defining custom ones, and there is a limitations page that most projects will find more useful than the feature list.

Session history, and the Rust story the tree contradicts

v0.16.0 added session recording, which changes the cost model. Results land under `.mutant/results/` and can be recalled without re-running:

# List past sessions (most recent first)
mutant session list

# Show full report from the latest session
mutant session show

# List subjects with alive/total mutation counts
mutant session subject

# Show alive mutations for a specific subject
mutant session subject 'Foo::Bar#baz'

# Remove old or incompatible session files
mutant session gc --keep 50

There is a published JSON schema for those session files, so recorded results are meant to be consumed by something other than the CLI. For a Rails project running mutant incrementally in CI, that is the difference between a gate you can query and a wall of text you scroll past.

Now the contradiction. The 0.15.1 release notes state that the mutant Rust wrapper stub was removed and that the Rust wrapper is cancelled in favor of a spiritual successor to mutant. Yet the repository tree still contains `Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml` and a `manager/` directory, and the Cargo manifest declares an active workspace:

[workspace]
resolver = "2"
members = ["manager"]

[workspace.package]
version = "0.0.1"
edition = "2024"

A version of 0.0.1 with edition 2024 tells you what this is: a scaffold, not the successor. The useful reading is that the fast rewrite is not in this repository, and anyone arriving from the crate names or the Cargo files will be looking for something that was deliberately deferred. If a Rust implementation is what you need, this is the wrong dependency to plan around.

Ruby versions, Rails support, and one stale row

Mutant runs on Linux and macOS. No Windows row appears anywhere in the support material, which is worth knowing before you build a Windows CI lane around it.

The support table has four rows, and one of them disagrees with the release notes:

| Version | Runtime | Syntax | Mutations | | ------- | ------- | ------ | --------- | | 3.2 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | | 3.3 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | | 3.4 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | | 4.0 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |

v0.16.0 lists `Remove Ruby 3.2 support` among its changes, alongside the Concord removal, the unparser relaxation and the bump to unparser 0.9.0. So the 3.2 row describes a state before the current release, and the other three rows are the ones to plan against. The pattern is predictable for a tool with four supported lines: the table lags the changelog by a release or two. Check the changelog when a version matters rather than trusting the table alone.

Rails is the case the project has clearly invested in, with a table covering 7.2, 8.0 and 8.1 and the claim that hook recipes are verified in CI on every non-EOL Rails version. The specific capability is parallel-worker database isolation on both PostgreSQL and SQLite, which is the feature that makes mutation testing practical inside a test suite that already parallelizes. There is a Rails guide covering setup, eager loading and per-worker isolation, plus a runnable CI-verified example in `rails_example/` and a plain `quick_start/` example. Note the dependency on eager loading: mutation testing needs to load the whole subject graph, so environments that rely on autoloading will need the documented setup rather than their existing test bootstrap.

Licensing: NOASSERTION in metadata, subscription in prose

The repository metadata reports the license as NOASSERTION, meaning the automated detector could not classify it. The README describes something quite different, and more specific:

Free for open source, using `--usage opensource` for public repositories. Commercial use requires a subscription, listed as $30 per month or $250 per year per developer, with a separate enterprise path by direct contact and a documented pricing page.

A `LICENSE` file exists in the tree, but its text is not part of the captured material, so the exact grant cannot be confirmed from what is here. That gap is the thing to close first if this touches commercial work, because the two signals you can see point in different directions: a detector that found nothing recognizable, and a README that describes a dual model with a per-developer fee. Read the file rather than inferring the terms from either.

The annual figure is worth doing the arithmetic on while you are there, since $250 per developer per year is roughly ten months of the monthly rate, which is a normal annual discount and not evidence of anything unusual. The one structural point that is unusual is that the charge attaches per developer rather than per repository or per run, so a team of thirty is a different decision from a team of three, and the enterprise route exists precisely because the self-serve price does not scale the way most tooling prices do.

The project is single-maintained, by Markus Schirp, with a Discord, a GitHub issue tracker with 124 open issues, 2,193 stars and 162 forks. The research paper behind it is IEEE-published and mutant is referenced in the Trail of Bits Ruby Security Field Guide, which is the strongest argument for the method being more than one person's preference.

How this fits an AI-assisted codebase

The README opens with a question aimed at a specific 2026 workflow: AI writes your code, AI writes your tests, but who tests the tests. It is a fair question and the tool answers it in a narrow way, so it is worth being precise about the scope of the answer.

Mutation testing verifies that tests detect semantic change. It does not verify that the tests assert the right thing on purpose, and it cannot tell a well-chosen assertion from a coincidentally discriminating one. A suite that happens to fail on every mutation mutant generates still leaves the question of whether those assertions match intent unanswered. The mutants being generated are code mutations, not test mutations, which is the standard limitation of the method across languages and worth stating plainly because the opening question invites a broader promise than that.

What it does give you is a mechanical answer to a question humans answer inconsistently: does this test pull its weight. That question gets expensive exactly when code volume rises faster than review capacity, which is the premise in the README. Applied narrowly to a subject under active change, in incremental mode in CI as the README suggests, the alive-mutation count becomes a signal with a direction rather than a number to negotiate.

Two practical notes for that workflow. Incremental mode is a separate documented mode with its own document, so the first run and the steady-state run behave differently in cost. And test selection, documented separately, is the lever that decides whether a per-commit run is affordable or a nightly run is your ceiling.

Editorial conclusion

Mutant is the rare quality tool where the argument for adoption is arithmetic rather than fashion. Line coverage answers whether a line ran; mutation testing answers whether a line mattered, and on code that an assistant wrote for you those are very different questions. The tool rewards a specific habit, which is treating each surviving mutation as a decision rather than a defect: either the mutation is a simplification worth keeping, or a behavior is untested and now you know which one. Budget for that, because a first run on an untested subject can surface dozens of alive mutations at once and the temptation will be to configure it away. Three details deserve a check before you commit. Ruby 3.2 appears in the support table but was dropped in the v0.16.0 notes, so test on 3.3 or newer. The repository license field reads NOASSERTION while the README sells a paid subscription, so read the LICENSE file yourself if this touches commercial work. And the Rust workspace in the tree is a stub whose version is 0.0.1, not the fast rewrite the Cargo files might suggest.

Frequently asked questions

Official sources

1. Issues
2. mbj/mutant on GitHub
3. README
4. Releases