flay: finding duplicated Ruby by structure rather than by text

A duplication finder that compares structure, not text

flay analyzes code for structural similarities. The key word is structural. A copy and paste that renames every variable, changes every literal, reformats the whitespace and swaps braces for do/end is still the same code to flay, and the tool is built to find exactly that.

The project description states the position plainly: differences in literal values, variable, class and method names, whitespace, programming style, and braces versus do/end are all ignored. That is the whole premise, and it is what separates flay from a text search for repeated lines. A line-oriented duplicate detector will match boilerplate that looks alike and miss duplication that has been reformatted or renamed, which is the usual state of duplication by the time anyone goes looking for it.

The tool is for Ruby, and it works by parsing source rather than by pattern matching. The requirement list names ruby_parser and sexp_processor, which is where the parse tree comes from. The output is a set of matches, each with a mass figure, and the documentation example shows a match in a `:defn` node with a mass of 184 and an identical match in a `:for` node with a mass of 144, which is the tool reporting the size of the duplicated structure rather than the number of lines it spans.

The project is old and it shows, in the good sense: the homepage is a personal site, the documentation is in rdoc format, and the last commit in this repository was on 2026-07-19. The repository is not archived, and there are no GitHub releases, so installation is by gem.

Mass, identical nodes and the N-way diff

The scoring is the part that needs explanation, because a list of duplicate locations tells you where to look but not whether the duplication is worth fixing.

Every match is reported with a mass. In the documented example the two method definitions in a CGI library file match with a mass of 184, and an identical pair of `for` loops match with a mass of 144. The mass is a measure of how much structure the matched code contains, so a high mass means a large duplicated body rather than a short repeated fragment. A second rule in the feature list says that flay adds a score multiplier to identical nodes, which means that when two fragments are not merely similar but identical, the reported score reflects that difference.

The `--diff` option produces an N-way diff of the code, which is the feature that turns a list of locations into something you can act on. The documented output interleaves the two versions of a matched method, labelled A and B, with the differing lines shown so that the shared shape is visible. In the example, the two methods differ only in the helper they call, `checkbox` against `radio_button`, and in the labels used inside them, while the surrounding structure is identical. Reading that side by side is how you decide whether the pair is a candidate for extraction or two legitimately similar things.

Because the diff is an N-way diff, it is not limited to two files. The report groups matches by node type, so the `:defn` matches and the `:for` matches are listed separately, which is how you can attack the largest group first.

Installing the gem and running it on one file

Installation is a gem install, as the project documentation shows:

sudo gem install flay

The four dependencies have to be in place, and the documentation lists them. ruby_parser and sexp_processor do the parsing, path_expander handles the file selection arguments, and ruby2ruby is a soft dependency, needed only if you want to use `--diff`.

That last point is easy to miss and it matters, because `--diff` is the option that makes the output useful. Without ruby2ruby, flay still finds and scores the matches, but it cannot render the side by side comparison.

The invocation the documentation demonstrates runs the tool on a single file with verbose output and the diff enabled:

% flay -v --diff ~/Work/svn/ruby/ruby_1_8/lib/cgi.rb

The tool prints a processing line for the file, then the matches grouped by node type, each with its mass, the file and line number of both sides, and, with `--diff`, the interleaved code.

For a real session you would start on one file, because the tool is fast enough to make that the natural loop, and then widen. The file selection machinery described below is what lets you widen without rewriting the command each time.

Conservative pruning, liberal pruning and fuzzy matching

Two options shape how aggressively flay reports. The feature list says the tool provides conservative, which is the default, and liberal pruning options. Conservative pruning keeps the reported set smaller and stricter about what counts as a meaningful match, and liberal pruning trades precision for recall, reporting more candidates.

This is the main judgement call in using the tool, and it is worth making deliberately on a sample rather than globally. A conservative run on a large codebase gives you a short list of structural clones that are probably worth acting on. A liberal run gives you a long list that includes near misses, which is useful when you are trying to understand the shape of the duplication in a codebase rather than fix it immediately. Neither mode is wrong, but they answer different questions, and the number of matches changes a lot between them.

The other option is `--fuzzy` duplication detection. The tool already ignores differences in names, literals and formatting, and fuzzy matching widens that further. The combination works well for finding duplication that has been deliberately disguised, which happens in codebases where a previous clean up effort renamed things rather than extracting them.

A third piece of the tuning is `.flayignore`. The feature list says flay skips files matched via patterns in `.flayignore`, which is in a subset format of `.gitignore`. That matters more than it looks, because duplication detectors and generated code do not mix: vendored libraries, machine written parsers and minified assets are all full of repetitive structure, and they will bury the results you care about under noise. Excluding them by pattern is the difference between a report you read and a report you regenerate.

Scanning a whole project with dir_arg and FlayTask

The project ships with path_expander, and the feature list describes what that buys you. A directory argument expands automatically, so you can point the tool at a directory instead of enumerating files. An `@file_of_args` form persists the arguments in a file, which is how you keep a long, careful file selection in version control and reuse it. And a `-path_to_subtract` option ignores intersecting subsets of files or directories, which is the other half of the noise problem: a shared library directory that happens to sit inside the tree you scanned can be subtracted from the results.

The combination of directory expansion, a persisted argument file and a subtract path is the practical recipe for a large codebase. The argument file means the selection is reviewable, the subtract path removes vendored trees, and `.flayignore` handles the rest by pattern.

For projects that already use Rake, the tool includes FlayTask. That is the intended way to wire duplication analysis into a build, and it is also the way to get a sense of whether a change made things worse. The point of a duplication check in a build is not the report on its own but the comparison between runs, which is where the mass numbers become a trend rather than a snapshot.

One naming detail is worth flagging for anyone searching for the persistence option. The feature list refers to a flay-persistent plugin for working across large or many projects, while the known extensions list names flay-persistence, described as persisting results across runs. The two names do not match, so check the actual gem name in the project's extension list before installing either.

The tool also has an editor integration story in progress rather than complete. The todo list mentions editor integration for emacs and textmate, and vim integration has started in a separate repository, flaying the current file on save, load or on command. None of that is in the core gem, so treat it as an extra rather than a feature.

A plugin system that reaches beyond Ruby

flay is language independent in design, through a plugin system, and the feature list says other languages can be flayed. The core ships with parsers for `.rb` and `.erb`, and the documentation states that JavaScript and others will be available separately.

The known extensions list makes that concrete. flay-actionpack uses the Rails ERB handler, flay-js processes JavaScript files, flay-haml flays HAML source, and flay-persistence persists results across runs for multi-project analysis.

This is the most interesting architectural decision in the project. The duplication detection logic is separated from the language front ends, so a new language means writing a parser adapter rather than a new analyser. It also means the tool can be pointed at the awkward parts of a mixed stack, where duplication is usually worst, because Rails ERB templates and HAML views are exactly the places where the same structure gets copied into a dozen view files.

The trade-off is that each extension is a separate gem with its own maintenance, and the core does not include them. The plugin surface also means results across languages are not necessarily comparable, since each parser decides what to normalise. If you plan to run flay over a mixed codebase, expect to treat the per language numbers separately rather than as one score.

Licence ambiguity, release history and when flay is the wrong tool

The licence is the murkiest part of the project documentation. The README states MIT, and includes the full MIT text with a copyright line for Ryan Davis, Seattle.rb. The repository's own licence metadata, however, does not identify a licence file at all, so the classification for the repository is unknown. For a tool you run locally against your own code that distinction rarely matters. If you intend to redistribute it, bundle it in an internal platform, or use it inside a commercial product, the absence of a recognised licence file is the thing to resolve, and reading the LICENSE text that the README quotes is the starting point rather than the end.

There are no GitHub releases in the repository, so there is no version history to consult and no changelog beyond the History.rdoc file at the top level. Installation is by gem, which means the version you get is whatever the gem source serves. The last push was on 2026-07-19, so the code is not abandoned, but it is not a project on a fast release cadence either, and the tool should be treated as a utility you pin rather than one you track.

The clearest case where flay is the wrong tool is a codebase where duplication is not the problem. If your file is long because it does a lot, no analyser will make it shorter. flay also tells you where two fragments are alike, not what to do about it, and in a young codebase the honest answer is often to extract a helper, which is a design decision rather than a mechanical one.

There is a second limitation in how it reports. The mass score is a proxy for size of duplicated structure, not for harm. Two copies of a ten line validation routine score lower than two copies of a two hundred line algorithm, and the second is the one that costs you when it drifts. A report sorted by mass is therefore a starting order, not a priority list.

The alternative to weigh is the opposite approach rather than a rival tool. A text based duplicate detector will find identical blocks that flay normalises away, for instance a long literal block or a copy of a configuration file. The difference in approach is exactly the normalisation: flay parses and compares structure, so it finds the duplicates a human would recognise as the same logic, and misses the ones that merely look alike. A team whose problem is copy pasted blocks of literal code should use a text search, and a team whose problem is the same ten line validation routine written six times should use flay.

Editorial conclusion

flay is worth adding to a Ruby maintenance workflow when you suspect copy and paste rather than a shared abstraction is behind a growing file, since it compares structure and therefore finds duplication that a text diff would miss. It is the wrong tool for a mixed language codebase beyond the shipped parsers, for a project that wants per-commit duplication gating, and for anyone who needs a maintained license file, since the repository metadata does not identify a license even though the README states MIT. Before adopting it, install the four documented dependencies, decide between conservative and liberal pruning on a sample run, and check that the .flayignore patterns exclude the generated and vendored code you would otherwise drown in.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. seattlerb/flay on GitHub