Wallaby.js: Running Your JavaScript Tests While You Type

The test run as part of the editing loop

Wallaby is a continuous testing tool for JavaScript, designed to integrate into the development environment. It automatically runs your tests as you code, providing real-time feedback on test results, code coverage and errors, so that problems surface while you are still looking at the code that caused them. That is the whole pitch, and it is a different proposition from a test watcher. A watcher re-runs the suite when a file changes and then tells you whether it passed, which still leaves a round trip between the edit and the result. Wallaby claims to be close to instant incremental test execution, which means the result is effectively part of the same view as the code. The topic tags for the repository put the same way, listing continuous-testing, test-runner and test-coverage among a set that includes tdd and bdd, which tells you the intended mental model is the test-first loop rather than a post-hoc check.

What the live feedback actually shows

The feature list is where the tool becomes concrete. Realtime feedback covers errors and any expressions in your code, so a failing expression is surfaced where it is written rather than in a separate panel. Realtime code coverage is updated as you type, and there is a code coverage explorer and a test explorer for navigating them. There is quick navigation to failing tests and to the source of an error, which is the piece that decides whether the loop is usable, because a failure you have to look up is a failure you have already lost context on. The value explorer is for viewing runtime values, and the compact and side-by-side diff views cover the case where the change you made moved something you did not intend. Running and focussing on selected tests is what keeps the loop fast when the suite is large, since you are not waiting for the two hundred tests you did not touch.

The time travel debugger and edit-and-continue

The distinguishing feature is the time travel debugger, with edit-and-continue and an interactive timeline. The readme leads with a demonstration of it, which is a reasonable choice for a capability that is hard to describe in prose. The idea is that you can move back through the states the program was in while you were editing, rather than re-running from a fixed point to reach the moment something went wrong. Edit-and-continue is the part that makes it usable in a live coding session, since you can change the code and carry on from the state you were in rather than restarting. This is the feature that separates Wallaby from a coverage overlay, and it is also the feature that makes the most demands on the runtime, since the tool has to keep more state than a straightforward re-run would need.

The editor and framework surface it covers

The topic tags for the repository are the best inventory of the integration surface available. On the editor side they include visual-studio, intellij, vscode, vscode-extension, sublime-text-3 and atom-editor, which is a spread across both the heavyweight IDEs and the lighter editors, and the presence of vscode-extension as a separate tag suggests the VS Code integration is delivered as an extension rather than a configuration. On the test framework side the tags name jest, mocha, jasmine and ava, alongside bdd and tdd, and the language side covers javascript, typescript and nodejs with webpack in the mix for bundler-based projects. None of that is spelled out in the readme itself, which is worth noting when you evaluate the tool, since the readme is a summary of the product rather than a specification of the integrations.

What this repository is, and what it is not

The repository description is Repository for Wallaby.js questions and issues, and the top level confirms it. There is a README, a CHANGELOG, a EULA.md, a .github directory, a .gitignore and a sample directory. There is no source code for the tool itself, no package.json for the product and no build configuration, so this is not where the software lives and it is not a place to send a pull request against the implementation. What it holds is the issue tracker and the questions, which is where the community asks about configurations, integrations and edge cases. The two sample directories, sample/jetbrains/ and sample/visualstudio/, suggest they carry sample projects for the two heavyweight IDE integrations, which is the kind of thing you would want when a configuration question comes up.

Where to get it and where the documentation is

The readme closes with three links and no install command, which is a deliberate choice for a commercial product rather than an open source package. The docs are at wallabyjs.com/docs, the website is at wallabyjs.com and the download is on a separate path at wallabyjs.com/download, with the site address in the repository metadata matching. The documentation links go further than a quick start would. There is a page on advanced logging for the realtime feedback, one on coverage, one on the time travel debugger, one on running selected tests, and the whole set sits behind a versioned docs site. The presence of an EULA.md alongside a CHANGELOG in a repository with no code is a reminder of what kind of project this is, and it is worth knowing before you plan to use it in a team.

How it compares with the runner you already have

The alternative is the standard loop of writing a test, running the suite from a terminal, and reading the output, which every JavaScript project already supports through its own runner. That approach has two advantages: there is nothing to install beyond the runner, and it works the same on a continuous integration server as it does on your laptop. Wallaby gives up both of those in exchange for the immediacy of the loop, since the tool has to run somewhere continuously and connect to the editor to report back. A concrete comparison to run before you commit is the one the topic tags suggest, since Jest, Mocha, Jasmine and AVA all behave differently in how they report and how long they take to start. If your suite is slow to boot, a tool whose value depends on a warm run will feel slower than the terminal, not faster.

Editorial conclusion

Wallaby fits a JavaScript or TypeScript developer who wants test results, coverage and errors updating as they type, and who is willing to install an editor extension and a small local server that runs the tests. It is the wrong choice if your suite is slow to start, since the value comes from the run being already warm, and the wrong choice if you need a CI-grade reporter rather than a development loop. Before you adopt it, check the docs for your editor and test framework, because the integration surface is where this tool lives or dies.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. wallabyjs/public on GitHub