Tesser: Clojure folds that run in parallel and on other machines

The same pipeline, with the reduce fanned out

The motivating example is deliberately ordinary, which is the point. You have a big pile of data, say JSON in files on disk or TSVs in Hadoop, and you want to reduce over it to compute some statistics or search for special values. The examples given are finding the median housing price in each city given a collection of all sales, finding the total mass of all main-sequence stars in a region of sky, or searching for an anticorrelation between vaccine use and the prevalence of a disease. These are all folds, which the documentation defines as collapsing a collection of data into a smaller value. In Clojure you would normally write it as a thread-first pipeline:

(->> stars
     (filter main-sequence?)
     (map :mass)
     (reduce +))

The complaint is not that this is wrong but that this reduction is singlethreaded and can only run on a single machine, when the machine in front of you has 48 cores. The same pipeline with Tesser looks almost identical:

(require '[tesser.core :as t])
(->> (t/filter main-sequence)
     (t/map :mass)
     (t/fold +)
     (t/tesser (t/chunk 1024 stars)))

Same shape, same readability, and now the work is chunked and parallel. The claim in the documentation is that Tesser goes much deeper, but this is the essence, writing understandable, composable, fast programs for exploring datasets.

A fold is six functions, and that is the whole model

The internal representation is worth understanding because it explains why the library composes. A compiled fold is a map of six functions, and the documentation gives the shape directly:

{:reducer-identity  (fn [] ...)
 :reducer           (fn [accumulator input] ...)
 :post-reducer      (fn [accumulator] ...)
 :combiner-identity (fn [] ...)
 :combiner          (fn [accumulator post-reducer-result] ...)
 :post-combiner     (fn [accumulator] ...)}

The two halves are symmetric. The reducer side needs an identity to start from, a step function to fold each input into the accumulator, and a post step to transform the result once the chunk is done, for example converting from a transient to a persistent data structure or discarding internal reducer state. The combiner side has the same three roles, and its accumulator is the post-reducer result of another chunk. The documentation notes that the reducer and combiner often have the same accumulator type and identities, but that this is not always the case, which is the point of separating them. A fold over a collection is therefore a two-level tree: chunks reduce in parallel, chunk results combine, and the combine step is the same shape of function as the reduce step. That symmetry is what lets folds be composed from smaller folds rather than written from scratch each time.

Most of the six keys can be omitted

The ergonomics come from the fact that most folds need only two of the six functions. A fold to sum all inputs can be written with everything spelled out:

(require '[tesser.core :as t])
(t/tesser [[1] [2 3]]
          (t/fold {:reducer-identity  (constantly 0)
                   :reducer           +
                   :post-reducer      identity
                   :combiner-identity (constantly 0)
                   :combiner          +
                   :post-combiner     identity}))
; => 6

And then it can be reduced. Since the addition function returns 0, the additive identity, the identities can be left off and t/fold will use the reducer as the identity function. The same applies to the post steps, because adding a value to a value returns the value, so the post-reducer and post-combiner can be left off as well:

(t/tesser [[1] [2 3]]
          (t/fold {:reducer  +
                   :combiner +}))
; => 6

And then there is the shortest form of all, passing a single function to t/fold, which is used for all six:

(t/tesser [[1] [2 3]] (t/fold +))
; => 6

The three forms compute the same thing, and the same machinery handles a fold over strings. The documentation shows the summing fold transformed into one that operates on string representations of numbers, using the map transform:

(->> (t/map read-string)
     (t/fold +)
     (t/tesser [["1" "2" "3"]]))
; => 6

That is the compositional claim in practice, since the transform is applied to the fold rather than to the input, and Tesser provides a rich library of fold transforms for building up complex folds out of simple, modular parts.

Associative and commutative, and what that costs you

The trade is stated directly, and it is the decision a user has to accept before anything else. Tesser folds are associative and commutative monoids, meaning they can be applied concurrently and they do not preserve order. The stated reason is that this reduces the need for coordination between reducers. Two named alternatives are held up as contrast. Algebird provides ordered distributed folds, and clojure.core.reducers preserves order as well, so the ecosystem already contains both answers. The documentation notes that not all concurrent folds provide ordered results, naming Algebird's ordered distributed folds as the contrast, and that Tesser does not preserve order right now, though it might change that someday. The other boundary is idempotence. Unlike CRDTs, Tesser's folds do not require it, and the documentation makes the distinction concrete: in Tesser, 1 + 1 is 2, not 1, because each input factors into the reduced value exactly once, though the order is up to Tesser. That is the right choice for statistics, where double counting a record would be wrong, but it also means a retried or duplicated input is not harmless in the way it would be in a CRDT, and a pipeline that can deliver the same element twice needs to deduplicate before the fold.

Plain threads rather than the forkjoin pool

The execution model is a small decision with a large performance consequence, and the documentation explains it. Applying a fold with tesser.core/tesser uses multiple threads proportional to the number of processor cores. Unlike reducers, it does not use the Java forkjoin pool, but plain old threads, and the stated reasons are that it avoids contention issues and improves performance on most JDKs. That is a real difference from the standard library behaviour, since clojure.core.reducers runs on the forkjoin pool, so a Tesser fold will not share that pool with other reducers in the same application. The consequence for ordering is visible in the example the documentation gives:

(require '[tesser.core :as t])

(t/tesser [[1 2 3] [4 5 6]] (t/into [] (t/map str)))
=> ["4" "5" "6" "1" "2" "3"]

The output is grouped by chunk and the chunks are in a different order from the input. Anyone porting a sequential pipeline to Tesser has to confront that here rather than in production, which is a mercy. The tesser.core namespace deliberately mirrors the shape of the Clojure sequence API, with many similarly named functions, but the semantics differ, since folds do not preserve order and run in parallel. That similarity is what makes the library readable to a Clojure programmer and also what makes it a trap: a function that looks like the seq version may have different behaviour, and the names are similar enough that the difference is easy to miss.

The quick and dirty entry point, and the two published artifacts

If all you need is a quick-and-dirty replacement for an existing reduce or fold, the documentation says to drop tesser.simple into your program, and the example is a single call:

(require '[tesser.simple :as s])
user=> (s/reduce + 0 (range 10000000))
  49999995000000

That namespace exists so that the migration from a sequential reduce does not require understanding the six-function representation first. The rest of the project is published as two artifacts on Clojars, both with API documentation. The core library and essential folds are com.aphyr.tesser.core, and tesser.math is a separate package of statistical folds covering means, correlations, covariance matrices, quantiles and similar. The separation is worth noticing, because the statistics package is the one most users want and it is a small, self-contained dependency, while the core library is the machinery that makes folds parallel and composable. There is one piece of administrative news to plan around. The dependency names are changing, with tesser.core and tesser.math moving to com.aphyr/tesser.core and com.aphyr/tesser.math, and the documentation is explicit that this is the same maintainer with a new Maven group name. The repository root also holds directories for the all, core, hadoop, math and img pieces, and the hadoop directory is what connects this to the motivating example of TSVs in Hadoop, which is the distributed case the library was designed for in the first place.

Editorial conclusion

Tesser is a good fit for a Clojure team that is already comfortable with the idea of a monoid and wants its statistics code to scale past a single core, especially when the work is a fold that can be expressed as a combine of combines. It is the wrong tool when order matters, since Tesser folds deliberately do not preserve order and the documentation says so plainly, and it is the wrong tool if you need idempotence, since the design explicitly is not a CRDT and each input factors in exactly once. Before adopting it, note the dependency rename, where tesser.core and tesser.math are moving to com.aphyr/tesser.core and com.aphyr/tesser.math with the same maintainer, and check the mathematical library separately if you need means, correlations, covariance matrices or quantiles. The repository publishes no GitHub releases, and the last push was on 2026-07-17.

Frequently asked questions

Official sources

1. aphyr/tesser on GitHub
2. Issues
3. README