BitFaster.Caching: bounded caches that behave like ConcurrentDictionary

The problem is a cache that has quietly outgrown its container

Most .NET services that need caching start with ConcurrentDictionary and never give it a size limit. That works fine until the key space is driven by something unbounded, a request id, a tenant id, a user id, and the dictionary becomes the largest thing in the process. BitFaster.Caching exists for that moment. The library describes itself as high performance, thread-safe in-memory caching primitives for .NET, and the primitives are deliberately close to the dictionary you already use. ConcurrentLru is presented as a lightweight drop in replacement for ConcurrentDictionary, with one difference: bounded size. You choose a capacity and the cache evicts when it reaches it. The pitch is that you get the same call shapes you already use, GetOrAdd and the rest, and you get a memory ceiling without rewriting call sites or introducing a hosted service to babysit an eviction timer.

ConcurrentLru evicts with a 2Q derived policy and no background threads

The eviction policy is the substance here, not the API. ConcurrentLru is a pseudo LRU based on the 2Q eviction policy, and the implementation detail that matters is what is absent. The README states there are no background threads and no global locks. There is no timer task sweeping the cache and no lock serializing every access, so concurrent throughput stays high and lookups stay fast. Time based eviction is offered as a separate option on the same cache, which means the entry age can matter independently of the admission and eviction rules. The hit rate claim is stated as the cache outperforming a pure LRU in all tested scenarios, which is the kind of statement that should be read as the library's own measurement rather than a general truth. The repository carries dedicated projects for hit rate analysis and throughput analysis, so the comparison work is in the source tree rather than only in prose.

ConcurrentLfu trades write latency for a near optimal hit rate

ConcurrentLfu is the other headline primitive, and it takes a different route to the same bounded goal. It is an approximate LFU based on the W-TinyLFU admission policy, and the described benefit is a near optimal hit rate with high scalability. The mechanism behind that scalability is the interesting part. Reads and writes are buffered and then replayed asynchronously, which is how the cache mitigates lock contention on the hot path. The cost is real and structural. Buffered writes mean a write is not immediately visible to a reader that arrives right after it, and asynchronous replay means the admission decisions that decide which entries survive are made on data that is slightly behind. For a cache where a hit is worth a database round trip, that latency trade is usually worth it. For code that relies on read-your-writes against a cache it just wrote, it is not, and ConcurrentLru is the safer choice because it does not defer writes.

Atomic valueFactory is the answer to a thundering herd

Cache stampede is the failure everyone who has used a cache has met. Many threads miss on the same hot key at the same moment, all of them go to the database, and the database gets hammered with identical work. BitFaster.Caching offers a configurable atomic valueFactory that mitigates this, and the word configurable is what makes it interesting rather than a convenience. The default GetOrAdd pattern lets every caller run the factory, so the stampede is present unless you change the factory. The atomic variant coordinates so that one caller does the work while the others wait or read the resulting value. The wiki page on atomic GetOrAdd is linked from the README as the place with the full API documentation, and it is worth reading before you assume the default behaviour is what you want. Note the scope of the mitigation. It removes the duplicate concurrent work for one key at one moment. It does not add a circuit breaker, a stale-while-revalidate policy or a timeout, so a factory that itself is slow will still hold a queue of waiting callers.

The builder API and the smaller pieces around the caches

Configuring several cache features one property at a time gets old, so the library ships a builder API for it. The README lists the features that can be assembled this way, and the list tells you what the library considers its building blocks. A configurable atomic valueFactory for stampede mitigation is one. Thread-safe wrappers for IDisposable cache values are another, and that is a specific problem worth naming: a cached object that holds a disposable resource, such as a stream or a handle, can be disposed by one caller while another is still using it, and the wrappers exist to stop that. SingletonCache is offered for caching single instance values such as lock objects, which is a way to share one synchronization primitive across code paths without a static field. There are also high performance concurrent counters in the metrics area, which are separate from the caches and can report on what the cache is doing.

Installing the package and using a cache for the first time

Installation is a single NuGet reference, and the getting started section gives the exact command.

dotnet add package BitFaster.Caching

A first cache is a capacity and a constructor, then the familiar dictionary call. The ConcurrentLru example in the README creates the cache with a capacity of 128 and reads a value with GetOrAdd, passing a factory that builds the item for a missing key.

int capacity = 128;
var lru = new ConcurrentLru<string, SomeItem>(capacity);

var value = lru.GetOrAdd("key", (key) => new SomeItem(key));

The ConcurrentLfu usage is identical apart from the type name, which is the point of the drop in claim. Pick the capacity first, because that number is the entire memory policy. The README also points to the wiki for full API documentation and for the complete analysis of hit rate, latency and throughput, and that analysis is where the choice between the two policies should be settled rather than in a blog post.

What this library does not do

The scope is narrow, and it is easier to state the boundaries than to guess at them. These are in-memory primitives in a single process. Nothing here persists across a restart, nothing is shared between machines, and there is no remote cache client, so a multi-instance service that needs a shared cache still needs one from somewhere else. There is no built-in expiry on the ConcurrentLru unless you configure the time based eviction option, so an entry can sit in the cache until capacity forces it out, and the choice of whether that matters depends on your key space. The repository also has no release notes in the README, and the recent tags are v2.6.1 from 2026-08-27, v2.6.0 from 2026-05-04 and v2.5.4 from 2025-06-28, so upgrade notes are not something you will find described in the project file itself. The library is MIT licensed, which is the permissive end of the scale and matters if you ship it inside a commercial product.

Editorial conclusion

BitFaster.Caching fits a .NET service where a ConcurrentDictionary has quietly grown into an unbounded memory liability and you want a ceiling with a defensible eviction policy, especially where read-heavy workloads make W-TinyLFU admission worth the write buffering. It does not fit a workload whose cache must be shared across processes or persisted across restarts, because these are in-memory primitives on one machine, and it does not fit a team that will not read the wiki, since the interesting trade-offs in the library live in its policy documentation rather than in its API surface. Before adopting it, install BitFaster.Caching from NuGet, measure your own hit rate and latency on the wiki's analysis projects, and confirm that the atomic valueFactory path is what you want for the keys several threads can miss at once.

Frequently asked questions

Official sources

1. bitfaster/BitFaster.Caching on GitHub
2. Issues
3. License: MIT
4. README
5. Releases