showyourwork: a workflow manager for reproducible scientific articles

The idea, stated in one paragraph

The readme is short, and the first substantive paragraph is where the whole project is described. showyourwork is a workflow management tool for open source scientific articles, and if you want to make your research article reproducible, extensible, transparent, or just downright awesome, you have found the right tool. It automates your entire workflow, packaging it into a self-contained recipe that anyone can follow and, at the click of a button, use to reproduce your results.

That is an unusual pitch for a scientific tool, because the problem it solves is not usually described as a tooling problem at all. A paper with figures has an implicit dependency: the reader is expected to believe that the figure was made by some script, and to trust that the script still works. Most papers do not make that dependency checkable, and the result is that the figure is only as credible as the reader's willingness to take it on faith.

The recipe framing is the important part. A recipe is a list of ingredients and instructions, and the point of packaging the workflow into one is that the reader does not need to know your directory layout, your environment, or the order in which you happened to run things. The tool has already encoded all of that, and a reader can follow it without understanding it, which is a lower bar than the one most reproducibility efforts implicitly set.

The claims are four, and they are worth separating. Reproducible means a reader can get the same outputs. Extensible means somebody else can add a step without rewriting the whole thing. Transparent means the intermediate products are visible rather than implied. And the fourth is a wink, because a workflow that produces a document anyone can rebuild is, in the readme's word, awesome.

The project is MIT licensed, the site is show-your.work, and the documentation is published separately at showyourwork.readthedocs.io. The primary language recorded for the repository is TeX, which is a clue worth holding on to: the manuscript is not an afterthought here, it is the centre of the thing.

The dependency list is the design

The package manifest is where the architecture of the tool becomes visible, and it is worth reading as a set of decisions rather than as a list.

The workflow engine is Snakemake, with a recent minimum version. Snakemake is a Python workflow engine built around rules and the files they produce, so the natural translation of a scientific pipeline into it is direct, and the reason it is here rather than a shell script or a Makefile is that the tool is about reproducibility, and a workflow engine that knows the dependency graph, can resume, and can report what it did is a better foundation for that than a shell script.

The next two entries are about the shape of a project rather than the running of a workflow. cookiecutter is the scaffolding tool, and it is what generates a new project directory from a template. And packaging is the library that knows how to compare version specifications, which is what you need if the tool is checking that a user's environment satisfies a requirement before it starts.

Jinja2 is the templating engine, and its presence tells you the tool generates text: configuration files, LaTeX fragments, and the parts of the manuscript that are not hand-written. Two YAML libraries are there, and they do different jobs. PyYAML is the parser. ruamel.yaml is the round-trip editor, the one that can read a YAML file, change one value, and write it back without discarding the comments and the formatting, and the fact that both are present is a sign that the tool edits user configuration rather than only reading it.

The remaining four are small and each one is a hint. graphviz is there to draw the dependency graph, so the tool can show you the shape of your workflow rather than only running it. requests is the HTTP client, and it is there for the downloads that a recipe needs, such as fetching data from a repository. click is the command line framework, which is what makes the single entry point pleasant to use. And pulp is a solver library for optimisation problems, which is what a tool that resolves version constraints for a user's environment would use.

None of these is exotic, and that is the point. The tool is a composition of well known pieces, and the value is in the composition.

One command, and the docs that tell you what it does

The package registers a single console script, and the entry point is a function in the command line module of the package.

[project.scripts]
showyourwork = "showyourwork.cli:entry_point"

A single command with subcommands is the right shape for this kind of tool, because the operations a user performs are distinct: creating a new project from the template, running the workflow, inspecting the results, and building the article. Each is a subcommand rather than a flag soup, and the readme points at a page in the documentation for the command line interface specifically.

The documentation structure is itself a map of the concepts, and the readme lists the pages: a quickstart tutorial that should get you up and running fairly quickly with a simple workflow, then pages for the command line interface, the recommended project layout, the workflow configuration file, and the details of the integration with LaTeX, and finally a list of example projects that use the tool.

That list is a good index of what a new user has to learn, and it is ordered by weight. The quickstart comes first because the tool has to be runnable before it can be understood. The layout comes before the configuration, which is the right order, because the configuration is expressed in terms of the layout. And the LaTeX integration comes last in the list but is arguably the most important part, because that is where the tool stops being a pipeline runner and becomes something about articles.

The example projects page is the other half of the documentation strategy. A tool like this is hard to describe in prose and easy to show in a directory tree, so the documentation points at real projects that use it, and the sample projects badge in the readme header counts the repositories that depend on the action rather than the library.

The project also says plainly that it is a work in progress, points at the issue tracker for bugs, invites new issues, welcomes community contributions, and mentions a new discussion board. The last release on record is version 0.4.3, with two release candidates before it, all in the same week in August 2023.

LaTeX, and why the manuscript is in the loop

The repository's primary language being TeX tells you something that the dependency list only hints at: in this project the article is part of the workflow rather than a thing that comes out of it.

Most pipelines for scientific articles end at a figure. The script makes a plot, the plot goes into a folder, and at some point a human drags the file into a document. That step is the reproducibility leak, because it is the one step that is not recorded anywhere and cannot be re-run. It is also where the silent errors live, because a figure in a document is a copy of a figure, and a copy can be stale.

The integration with LaTeX is what closes that leak. If the manuscript is assembled by the same system that runs the computation, then the document depends on the outputs of the workflow, and re-running the workflow re-builds the document with the current outputs in it. The failure mode of a stale figure stops being possible in the same way a build artefact that is out of date stops being possible in a compiled program.

The tooling choices around it are consistent with that goal. The documentation set for the project itself uses an extension that captures program output, so that the commands a user runs and what they printed appear in the manual, which is the same idea applied to the documentation. The documentation also includes a theme, a design extension for landing pages, and a changelog extension, which is a list that says what the tool is expected to be used for and by whom.

And the sample projects are the third piece. A user who wants to know what a finished article repository looks like can read one, and the existence of a public gallery of them is what turns the idea from a proposal into a convention.

The one thing to be careful about is the age of the release. The last tagged version is from 2023, while the workflow engine it depends on is required at a much more recent version than that. In other words, the tool is old and the requirement is new, which is a combination that tends to mean the package will install and the project will not build until you do something about it.

Packaging, versioning, and the test setup

The build configuration has a few details that tell you how the project is maintained.

The build backend is hatchling with the version control plugin, and the version is taken from the source control tags rather than being written down. The plugin writes the resolved version into a generated module inside the package, which is the arrangement that lets a package report the exact commit it was built from without anybody having to remember to bump a number.

That matters for a reproducibility tool in particular, because the version of the tool is part of the record of how a result was produced, and a version that is derived from the tag is a version that cannot drift from the commit.

The project also requires a recent Python, at least 3.11, and the classifiers list 3.11, 3.12, and 3.13, and it marks itself as production and stable. The optional dependency groups are three, and they are well chosen. The tests group adds the plotting library and the test framework, including an extension that runs asynchronous tests cooperatively, which is the right tool for a package whose tests involve workflows that wait on other processes. The documentation group adds the theme and the extensions, and the development group combines the other two with the linter and the pre-commit hooks. That structure means a contributor installs one extra group and gets everything they need.

The test configuration shows the same attention. The default options are verbose, and they set a task timeout of half an hour, which is generous for a unit test and is exactly the number you would pick for a test that starts a workflow and waits for it. The tests directory is excluded from the recursive search in one place, and that place is a helpers directory, which is a sign that the helper code is a package of its own rather than a set of test modules.

The repository layout is tidy in a way that shows the tooling. There is a Ruff configuration, a pre-commit configuration, a towncrier configuration for the changelog, a changes file, a history file, a citation file for the academic citation of the software itself, and a Read the Docs configuration.

Editorial conclusion

showyourwork fits a research group whose figures are produced by scripts and whose reviewers or readers need to be able to re-run them, and it fits a solo researcher who has never been asked how a figure was made. It does not fit a paper that is only text, and it does not fit a group that wants a hosted service rather than a repository. Verify first that your article is written in LaTeX, since the integration with the manuscript is the part that does the work, that the scripts you want to include are already in the shape the tool expects rather than a notebook, and that the version of the tool you pin still matches the version of the workflow engine it requires, because the last release on record is from 2023 and the engine requirement is recent.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. showyourwork/showyourwork on GitHub