async-lru caches coroutine results and collapses concurrent calls

One underlying call for many concurrent callers

The behaviour that separates async-lru from a plain decorator is stated in one sentence in the README: it ensures multiple concurrent calls will only result in 1 call to the wrapped function, with all awaits receiving the result of that call when it completes.

That is a single-flight guarantee, and it is the reason to reach for this library rather than to hand-roll a dictionary of futures. If ten requests arrive for the same uncached key while the wrapped function is still running, the naive implementation performs ten round trips and caches ten results. Here, one call happens and the other nine wait for it.

The package is a port of Python's built-in functools.lru_cache function for asyncio, which means the familiar surface carries over. The decorator takes maxsize, the wrapped function gains a cache_info() method that reports hits, misses, maxsize and currsize, and the eviction order is least recently used as the name suggests.

The shape of the API is worth reading closely:

from async_lru import alru_cache

@alru_cache(maxsize=32)
async def get_pep(num):
    ...

One thing the README asks you not to skip is the cleanup. The example calls await get_pep.cache_close() at the end, with the comment that closing is optional but highly recommended. A cache holds tasks and connections associated with in-flight calls, and the library gives you an explicit way to release them rather than leaving it to garbage collection timing.

TTL with jitter to avoid a thundering herd

Expiration is supported through a ttl configuration parameter, which is off by default. The default matters, because a cache with no expiry and a bounded maxsize behaves like the standard library decorator and nothing more.

With a TTL set, entries expire after the given number of seconds:

@alru_cache(ttl=5)
async def func(arg):
    return arg * 2

The jitter parameter exists to solve a specific failure mode. The README explains that to prevent thundering herd issues when many cache entries expire simultaneously, you can add jitter to randomize the TTL for each entry:

@alru_cache(ttl=3600, jitter=1800)
async def func(arg):
    return arg * 2

With those values, each cache entry gets a random TTL between 3600 and 5400 seconds, which spreads out invalidations over time instead of concentrating them at one moment. The scenario is the familiar one where a cache is populated by a batch of similar requests, every entry shares the same lifetime, and they all lapse at the same instant and hit the origin together.

The trade-off is worth stating plainly. Jitter trades predictability of expiry for spread of load, and it means the age of any given entry is not knowable in advance. If your data has a hard correctness requirement, such as a value that must not outlive a permission by even a second, a randomised TTL is the wrong instrument and a short fixed one is safer.

Invalidation and membership checks are separate operations

The library exposes two operations that are easy to confuse and are deliberately distinct.

cache_invalidate() removes a specific function call from the cache. It takes the same arguments the wrapped function takes, and it returns True if the corresponding argument set was cached already and False otherwise. That return value makes it usable as a test rather than only as a mutation:

@alru_cache(ttl=5)
async def func(arg1, arg2):
    return arg1 + arg2

func.cache_invalidate(1, arg2=2)

cache_contains() answers whether a specific set of arguments is present, and the README is explicit about its contract: it does not affect the hit and miss counters or the LRU ordering. That distinction is the point of having it. Checking whether something is cached is a read, and a read that reorders the cache or increments a counter changes the behaviour of the cache under observation:

await func(1, arg2=2)

func.cache_contains(1, arg2=2)  # True
func.cache_contains(3, arg2=4)  # False

This is the pattern you want before a read-through path that should not populate the cache, or in a health check that reports whether a warm value exists without perturbing the eviction order.

Custom keys go beyond the functools interface

By default the cache key is built from all arguments, the same way functools.lru_cache does. The key parameter accepts a callable that receives the same arguments as the wrapped function and returns the cache key, so arguments that do not affect the result can be left out. The README calls this an async-lru extension beyond the functools.lru_cache interface.

The motivation in the example is a database connection that is not part of the answer:

@alru_cache(key=lambda db, query: query)
async def query_db(db, query):
    return await db.execute(query)

# Both calls share one cache entry despite the different connections.
await query_db(conn1, "SELECT ...")
await query_db(conn2, "SELECT ...")

Without the custom key, each connection object would produce a separate entry and the cache would fill with duplicate results that differ only by an object identity nobody cares about.

Two constraints come with it. The returned key must be hashable. And cache_invalidate() and cache_contains() compute the key the same way, so they accept the full argument list as usual rather than the reduced key. Passing typed=True together with key raises a ValueError, because typed only affects the default key computation and would have no meaning once you supply your own.

For decorated methods there is one more detail. The key callable receives the instance as its first argument, so a method key function has to account for self.

Event loop affinity will raise at call time

The first of the two documented limitations is a hard failure rather than a subtle bug, and it is the one that will bite you in a test suite or a threaded application.

alru_cache enforces that a cache instance is used with only one event loop. If you attempt to use a cached function from a different event loop than where it was first called, a RuntimeError is raised:

RuntimeError: alru_cache is not safe to use across event loops: this cache
instance was first used with a different event loop.
Use separate cache instances per event loop.

The README says that for typical asyncio applications using a single event loop this is automatic and requires no configuration, which is true. The problem is code that creates loops on the fly, and code that runs in more than one thread, where the same decorated function ends up reachable from two loops.

The documented remedy is a separate cache instance per loop, and the example uses a thread local to hold the decorated function:

_local = threading.local()

def get_cached_fetcher():
    if not hasattr(_local, 'fetcher'):
        @alru_cache(maxsize=100)
        async def fetch_data(key):
            ...
        _local.fetcher = fetch_data
    return _local.fetcher

There is a second route for reusing the logic of an already decorated function in a new loop, by accessing __wrapped__ and re-decorating it, which creates a fresh cache instance for the same underlying function:

cached_task_loop2 = alru_cache(maxsize=32)(my_task.__wrapped__)

The deeper point is that the restriction is not arbitrary. A cache holding pending awaitables is bound to the loop that will resume them, so the library refuses rather than misbehaving quietly.

The cache key ignores request scoped context

The second limitation is the one to read carefully before putting this in front of real users, and the README puts it under a heading of its own, security considerations.

Cache keys are built only from explicit arguments. Like functools.lru_cache, alru_cache derives its cache key solely from the positional and keyword arguments passed to the wrapped function. Implicit, request-scoped context, such as authentication headers, the current user or tenant, contextvars, thread or task locals, or module globals, is not part of the key and therefore not isolated between callers.

Stated plainly, a cached function that reads the current user's permissions from a context variable will return the first user's result to the second user if the arguments match. That is a cross-tenant data leak, and it is a property of the design rather than a bug to be filed. The whole family of decorators that key on arguments has this property, which is why the documentation is explicit about it.

There are two ways to handle it and both are your decision to make. The first is to include the relevant context in the arguments, so the key distinguishes callers. The second is to not cache those calls at all, and to use the custom key callable only when you have verified that everything which changes the result is either in the arguments or intentionally excluded.

The trap is the second path, because excluding an argument from the key is exactly what the key parameter is for, and it is easy to exclude something that turns out to matter. The README's warning is the reason to read this section before the API reference.

Tooling, packaging and the release cadence

This is a small library with the surrounding scaffolding of a maintained one.

The package is published on PyPI as async-lru, and the repository belongs to the aio-libs organisation, which also runs Matrix rooms for discussion and a wider community space. The README carries CI status badges for the GitHub Actions workflow and for code coverage through codecov.

The layout shows a conventional Python project. setup.py is a two-line shim that calls setup() with all configuration in setup.cfg, alongside MANIFEST.in, requirements.txt, requirements-dev.txt, requirements-test.txt and tox.ini, with the test dependencies pinned, including coverage and pytest-cov. The Makefile exposes the daily tasks: init and setup install the development requirements and install the pre-commit hooks, fmt runs pre-commit over all files, lint runs fmt and then mypy, and test runs pytest against the tests directory. A benchmark.py file with its own requirements-benchmarks.txt is present at the top level, so performance work has a home even though the README makes no performance claims.

The release history shows steady work. Version 2.4.0 was published on 2026-10-12, following 2.3.0 on 2026-03-19 and 2.2.0 on 2026-02-20, with CHANGES.rst recording the history. The last push to the repository was on 2026-10-18, and the repository is not archived.

The licence is MIT, which is permissive enough that the dependency is easy to take on in a commercial project. The one cost is the one described above, the need to reason about what belongs in the key.

Editorial conclusion

async-lru suits asyncio applications that call a slow async resource repeatedly, such as an HTTP client or a database, and that need the result cached with the eviction and invalidation controls the standard library decorator does not offer for coroutines. It is the wrong tool when your cache key would need to include the current user or tenant, because the key is built only from explicit arguments and request scoped context is not isolated between callers, and the wrong tool for code that runs the same decorated function across several event loops. Before you adopt it, decide whether your key function captures everything that changes the result, close the cache with cache_close() where you used the pattern from the README, and check whether your deployment ever runs more than one event loop, because that raises a RuntimeError at call time.

Frequently asked questions

Official sources

1. aio-libs/async-lru on GitHub
2. License: MIT
3. Project website
4. README
5. Releases