ScalaTest, a testing toolkit where each style is its own module

What the project is, and what the README chose to document

ScalaTest is a free, open-source testing toolkit for Scala and Java programmers, with the official website at scalatest.org. The project's own description of its standing is that it has been a cornerstone of testing in the Scala ecosystem for over 17 years, and that it is trusted by countless developers and teams to write expressive, flexible and robust tests.

It also states its own position on funding. The project has always believed in keeping ScalaTest free and open source, but maintaining a tool used so widely takes time, care and ongoing effort, and there is a sponsorship link for anyone who wants to support the work.

What makes this README unusual is what it does not contain. There is no feature list, no catalogue of matchers, no example of a test written in any style. Instead, the documentation covers the setup by pointing at the sbt Download and Setup page, the quick start by pointing at the quick_start page on the website, and then spends its length on building the project, running the tests, and publishing to Sonatype.

That is a deliberate division. The usage documentation lives on the website, and this README is the maintainer document. So if you are evaluating ScalaTest as a user, the README tells you very little, and if you are evaluating it as a potential contributor or a fork, the README is unusually complete.

The repository layout backs that reading. There is a build.sbt at the root, a project/ directory for the sbt build definition, jvm/, js/ and native/ directories for the three platform targets, dotty/ for the Scala 3 line, examples/, scalatest-doc/ for the documentation, lib/ and support/ for shared material, and a pile of files that say what they are, with todo.txt, notes/, migration.txt, addmem.sh and updateCopyright.bash sitting next to the build definition.

The release tags are release-3.2.20 from 2026-03-26, release-3.2.19 from 2024-06-26 and release-3.2.18 from 2024-02-08, the default branch is main, the last push was on 2026-10-12, and the licence is Apache-2.0 with a LICENSE file and a NOTICE file at the root.

One published module per test style, and why that shows up in the build

The most informative thing in the repository is the module list, because it reveals the design. The publishing section lists the modules that get a binary compatibility check, and each one is a test style as a separate artifact: scalatestFeatureSpec, scalatestFlatSpec, scalatestFreeSpec, scalatestFunSuite, scalatestFunSpec, scalatestPropSpec, scalatestRefSpec, scalatestWordSpec and scalatestDiagrams. Alongside them are the matcher modules scalatestShouldMatchers and scalatestMustMatchers, and the core modules scalactic and scalatestCore.

So the choice of test style is also a choice of dependency. A project that uses the word should style pulls in scalatestShouldMatchers, one that uses the must style pulls in scalatestMustMatchers, and a project that mixes styles pulls in several. The repository has separate JS variants of the same modules, with scalacticJS and scalatestCoreJS among the names in the publishing list, which is the same design applied to the Scala.js target.

The point of splitting this way is that each module is a small, separately versioned artifact, and each one can be checked for binary compatibility independently. That is what the publishing section is really about, and it is a good example of a library treating compatibility as a per-artifact concern.

Two supporting directories in the root reinforce the scale of the thing. The osgi.bnd and scalactic-osgi.bnd files are OSGi bundle descriptors, so the artifacts can be consumed in an OSGi container, and the pom_template.xml and scalactic-pom_template.xml are templates for generating Maven POMs, which is what the Sonatype publishing path needs.

The ant-lib/ and build.xml files are older build machinery alongside the sbt build, and the .jvmopts, .sbtrepos and .travis.yml files show a project that has been through a few CI generations.

Running the tests, and the three kinds of suite that need special handling

The build prerequisites are JDK 8 and SBT 1.3.13, and the README gives specific JVM options to use instead of the defaults:

SBT_OPTS="-Xms512M -Xmx3000M -Xss1M -XX:NewRatio=8"

The regular test run is a single command:

`$ sbt test`

Then there are the three suites that are not part of the default run, and each has a documented way to enable it.

Flicker tests need an environment variable:

> export SCALATEST_RUN_FLICKER_TESTS=true
> sbt test

The generated tests, which are the ones that check the library's own test generation, need a bigger heap and a different project:

$ rm -rf gentests
$ sbt gentests/test

And those generated tests can be run in groups, with a separate sbt invocation for each group of matchers, inspectors, they-style, contain, sorted, lone element and empty. The README explains that what it does is simply switch to the gentests project and run test.

That group list is worth reading as a map of the library's surface, because each generated group corresponds to a family of matchers. It is also a warning about the cost of the library: a project whose test suite takes a specific heap size and a specific set of environments to run is a project with a large surface, and a team adopting it inherits that.

For the cross-platform targets, the same pattern appears again. The Scala.js tests are run per project, with a clean, then a compile and a test for the scalactic and scalatest JS test projects separately:

$ sbt scalatestAppJS/clean
$ sbt scalacticTestJS/test:compile
$ sbt scalacticTestJS/test
$ sbt scalatestTestJS/test:compile
$ sbt scalatestTestJS/test

The Scala Native tests are the same shape with a Scala version switch and a property:

$ sbt -Dscalatest.skip.jdk.check=true ++2.11.12 scalatestAppNative/clean
$ sbt -Dscalatest.skip.jdk.check=true ++2.11.12 scalacticTestNative/test:compile

The skip.jdk.check property exists because the Native build pins an old JDK check, and the Scala version pin of 2.11.12 is there because that is the version the native target is built against. Reading those two flags together tells you how far the cross-platform support lags the JVM build.

Publishing, and the MiMa check that stands in front of it

The publishing section is the part of the README that has aged least, because the Sonatype process has not changed much, and it starts with the requirements: a GPG client on the command line path, GPG keys created with the public key distributed to a keyserver, and the right to publish using the org.scalatest and org.scalactic domains.

Credentials come from a properties file at ~/.ivy2/.credentials by default:

realm=Sonatype Nexus Repository Manager host=oss.sonatype.org user=xxxxxxxx password=xxxxxxxx

And the SCALATEST_NEXUS_LOGIN and SCALATEST_NEXUS_PASSWORD environment variables can override them. Signing uses ~/.gnupg/secring.gpg by default and prompts for the passphase if required, with SCALATEST_GPG_FILE to use a different GPG file and SCALATEST_GPG_PASSPHASE to provide the passphrase and avoid the prompt. There is also a documented command to export a private key into a separate file, which is what you would do to keep the release key out of the default location.

The sentence that matters most comes right after: before publishing any patch release, binary compatibility with previous version should be checked. And then the list of modules, one per artifact, each running scalactic or scalatestCore and then mimaReportBinaryIssues for the specific module:

$ sbt ++2.11.12 scalactic/package scalactic/mimaReportBinaryIssues $ sbt ++2.11.12 scalatestCore/package scalatestCore/mimaReportBinaryIssues

MiMa is the binary compatibility checker, and the fact that it is invoked per module and gated on patch releases tells you how the project treats its public surface. It also tells you where the API is: a module that is not in this list is not covered by the compatibility promise, and that list is a useful map for anyone depending on ScalaTest.

The other build commands in the README are more conventional. sbt examples/compile builds the examples project, sbt package produces the JAR in target/scala-2.11/, sbt publishLocal publishes to the local Ivy repository, and sbt publishM2 publishes to the local Maven repository.

Editorial conclusion

ScalaTest suits a Scala team that wants expressive, flexible tests and has more than one style of test in the same project, since the styles are separate artifacts and a team can mix them. It is the wrong choice for a Java-only project that wants a single convention, because the toolkit ships many styles and the choice is yours to make, and it is the wrong choice if you need a modern build, because the README's own instructions still target JDK 8, SBT 1.3.13 and Scala 2.11. Before adopting it, check which modules you actually need, since the artifact list is long, and if you are building from source, read the MiMa section, because binary compatibility is checked per module before any patch release and that list is the map of what the project considers public API.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. scalatest/scalatest on GitHub