maypok86/otter: an in-memory cache for Go built on adaptive W-TinyLFU

What otter solves, and which Go services it fits

Most Go services that cache something in process start with a map plus a mutex, then discover that the map never shrinks and that hot keys are indistinguishable from cold ones. Otter replaces that map with a bounded cache whose eviction decision is made by adaptive W-TinyLFU, the policy the README credits to the paper linked under its features list. The library also takes design cues from Caffeine, the Java cache, which the README names directly.

The intended user is a Go engineer running a service that reads from a slower source (a database, an HTTP API, a file) and wants to keep recent results in memory. The README lists the optional behaviours it supports: size-based eviction, time-based expiration measured since last access or last write, automatic loading, asynchronous refresh when the first stale request arrives, writes propagated to an external resource, access statistics, and saving the cache to a file and loading it back. Each of these is opt-in, and the README states that a cache can be configured as a plain hash table wrapper with near-zero memory overhead for features you do not use.

That last claim is the interesting design decision. Rather than shipping one generic node type that carries fields for every feature, the project generates node code through cmd/generator, which the README links as the reason unused features cost nothing. The trade-off is that the repository carries a code generation step in its Makefile, so anyone building from source or modifying internals has to run it.

How the eviction policy and configuration struct work together

Otter exposes a plain Options struct rather than a builder chain. The README points at otter.Options for the full field list and shows a partial example with MaximumSize, ExpiryCalculator, RefreshCalculator and StatsRecorder. Those four fields map onto four separate mechanisms.

MaximumSize sets the bound that triggers size-based eviction. ExpiryCalculator decides when an entry is considered expired; the README example uses otter.ExpiryAccessing, which resets the timer on reads and writes, and the feature list notes that expiration can instead be measured since last write. RefreshCalculator decides when an entry should be refreshed in the background; the example uses otter.RefreshWriting, described as refresh after writes. StatsRecorder receives access statistics, and the example attaches stats.NewCounter().

The repository layout backs this up. There are separate files for expiry_calculator.go, refresh_calculator.go, policy.go and sketch.go, plus a singleflight.go. The sketch file is consistent with the frequency estimation that TinyLFU depends on, and singleflight is the mechanism behind the stampede protection the README demonstrates: it shows concurrent Get calls deduplicating loader calls so that a slow load runs once rather than once per caller. The README's example also shows that a Get which triggers a background refresh returns the current value, not the refreshed one, which is the behaviour you want when you cannot afford to block a request on a reload.

Installing otter and running a first cache

The README requires Go 1.24 or above, and the go.mod file confirms go 1.24.0. It also notes that otter only supports the two most recent minor versions of Go, so a toolchain older than that is outside the supported range even if it compiles.

Installation is a single go get. The README gives separate commands for the v1 and v2 module paths, and states that v2 is the latest stable major version. Install v2 like this:

go get -u github.com/maypok86/otter/v2

After that, construct a cache with otter.Must and an Options value. The README's example sets a capacity of 10,000 entries, a one-second expiry since last access, a 500ms refresh interval after writes, and a stats counter:

cache := otter.Must(&otter.Options[string, string]{
    MaximumSize:       10_000,
    ExpiryCalculator:  otter.ExpiryAccessing[string, string](time.Second),
    RefreshCalculator: otter.RefreshWriting[string, string](500 * time.Millisecond),
    StatsRecorder:     counter,
})

With that cache in hand, cache.Set writes a value and cache.GetIfPresent reads one without triggering a load. The README example sets "key" to "value", sleeps past the one-second expiry, and then expects GetIfPresent to report the entry as absent. To load on miss instead, pass a loader to Get. The README wraps a plain function with otter.LoaderFunc and calls cache.Get(ctx, "key", otter.LoaderFunc[string, string](loader)), which returns the loaded value and an error. In the example the loader sleeps 200ms, which is how the README illustrates that concurrent Gets share one load rather than each starting their own.

Where otter is the wrong choice

Otter is an in-memory, single-process cache. Nothing in the README describes a network protocol, a client-server mode, or coordination between instances, and the repository has no server binary among its top-level entries. If two replicas of your service each hold their own otter cache, they will diverge: an invalidation on one replica does not reach the other. The README does list persistence to a file and loading from a file, but that is a snapshot mechanism, not a shared store, and it does not give you cross-process consistency.

The Go version floor is a second constraint. Go 1.24 or above is required, and only the two most recent minor versions are supported. A service pinned to an older toolchain, or one that builds across a matrix of Go versions, may find the supported window narrower than its own.

The feature set also carries configuration cost. Expiry, refresh, loading and stats are independent calculators and recorders that you assemble yourself. The README shows the combination but does not walk through what happens when expiry and refresh intervals interact, and the user guide is where that detail lives. A team that wants one obvious default and no tuning will spend more time reading documentation than it expected.

Otter compared with a plain map plus sync.Map

The most common alternative in Go is not another library but the standard library: a map guarded by a mutex, or sync.Map for read-heavy access patterns. The difference is not speed in the abstract, it is what happens when the cache fills up. A map has no eviction policy at all, so either you grow until the process is killed or you write your own eviction, and a naive random or FIFO eviction will keep entries that are never read again while discarding entries that are read constantly.

Otter's answer is adaptive W-TinyLFU, which the README links to a paper and describes as the source of high hit rates across workload types. The adaptivity matters because the policy adjusts to the workload rather than requiring you to pick a fixed strategy up front. The README also states that data structures are configured automatically based on contention, parallelism and workload patterns, so you are not choosing between a lock-free and a locked variant by hand.

Where sync.Map still wins is simplicity and dependency count. Otter's go.mod pulls in stretchr/testify plus its indirect dependencies, and the module requires Go 1.24. If your cache holds a few hundred entries that live for the lifetime of the process, an eviction policy is overhead you will never exercise.

Maintenance, licence and upgrade path

The repository is not archived. Its last push was on 2026-06-19, and the most recent release listed is v2.3.0 from 2025-12-22, preceded by v2.2.1 and v2.2.0 in July 2025. The project publishes a changelog and release notes, and the README directs readers to the release notes for details of the changes between major versions.

Otter is licensed under Apache-2.0, and the LICENSE file sits at the repository root. Apache-2.0 is a permissive licence that includes an explicit patent grant and requires preservation of notices; it is not a copyleft licence. That is a general description of the licence text, not legal advice, and anyone embedding the library in a distributed product should have their own counsel review the NOTICE and attribution requirements.

The upgrade story is versioned by module path. v1 and v2 are separate import paths, so upgrading is a change of import from github.com/maypok86/otter to github.com/maypok86/otter/v2 rather than a drop-in patch. The README states that v2 is the latest stable major version and that otter follows semantic versioning for the documented public API on stable releases, which means breaking changes within v2 should be reserved for major releases. The README does not document rollback or downgrade procedures, so a team moving from v1 to v2 should treat the release notes as the authoritative list of behaviour changes.

Editorial conclusion

Adopt otter when you want a single-process cache in Go with a tunable eviction policy and optional loading, refresh and persistence, and you can commit to Go 1.24 or later. Skip it if you need a distributed cache shared across processes, or if you are pinned to an older Go toolchain. Before adding it, read the v2 user guide for the feature you intend to enable, since the README shows only a partial Options struct, and confirm that the two most recent Go minor versions cover your build image.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. maypok86/otter on GitHub
3. Project website
4. README
5. Releases