# identity_cache: blob-level read-through caching for ActiveRecord

> Shopify's cache layer, extracted from production, that stores whole model objects in Memcached and replaces `find` with `fetch`, with secondary indexes, embedded associations and automatic invalidation through an `after_commit` hook. The interesting parts are the configuration comments about consistency, which are the library telling you what it is willing to trade.

**Shopify/identity_cache** — IdentityCache is a blob level caching solution to plug into Active Record. Don't #find, #fetch!

- Repository: https://github.com/Shopify/identity_cache
- Website: http://shopify.github.io/identity_cache
- Stars: 1,970 · Forks: 175
- Language: Ruby
- License: MIT
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/shopify-identity-cache

## What the library replaces, and where it came from

IdentityCache is described as a blob level caching solution to plug into Active Record, with the summary line: don't `find`, `fetch`.

The fuller description is more precise. It is opt-in read-through ActiveRecord caching used in production and extracted from Shopify. It lets you specify how you want to cache your model objects, at the model level, and adds a number of convenience methods for accessing those objects through the cache. Memcached is used as the backend cache store, and the database is only hit when a copy of the object cannot be found in Memcached.

So the unit of caching is the object, not the row and not the query result. A cached product is a single blob containing the product and whatever associations you asked to embed in it. That is the design decision everything else follows from, and it is the opposite of the usual approach of caching query results.

Invalidation is the other half. IdentityCache keeps track of the objects that have cached indexes and uses an `after_commit` hook to expire those objects, and any up the tree, when they are changed.

The up-the-tree part is the detail that matters. If a Product caches its Images, and an Image changes, the Image's own cache entry is expired and so is the Product that embedded it. Without that, a cached parent would keep serving a stale child indefinitely.

The project is MIT licensed, the primary language is Ruby, the default branch is main, and the repository is not archived with the last push on 2026-08-24. The recent releases are v1.6.5 from 2026-08-25, whose title is about properly passing MemCacheStoreCAS serialized entries to Dalli, then v1.6.4 and v1.6.3 from 2026-01-09.

## The configuration, and the comments that state the trade-off

The install adds the gem to your Gemfile, and there are two optional companions:

```ruby
gem 'identity_cache'
gem 'cityhash'        # optional, for faster hashing (C-Ruby only)

gem 'dalli' # To use :mem_cache_store
# alternatively
gem 'memcached_store' # to use the old libmemcached based client
```

The cityhash line is a performance option restricted to C-Ruby, which tells you the hashing is done in Ruby on other interpreters, and the two memcached clients are the old libmemcached one and the newer Dalli one.

The configuration itself is a cache store name, a list of servers and a set of options, and the options carry comments that say more than the values do:

```ruby
config.identity_cache_store = :mem_cache_store, "mem1.server.com", "mem2.server.com", {
  expires_in: 6.hours.to_i, # in case of network errors when sending a cache invalidation
  failover: false, # avoids more cache consistency issues
}
```

Two comments, two decisions. The expiry is there to cover the case where a cache invalidation fails because of a network error, so an entry lives at most six hours rather than forever. And the failover is off, and the reason given is that failover causes more cache consistency issues, because with failover enabled a client that loses a server silently talks to another one and the cluster ends up with two views of the same key.

The old client has the same shape with different names, where the second option is a configured Memcached client with `support_cas: true` and `auto_eject_hosts: false`, with the same comment about avoiding more cache consistency issues.

In both cases the backend is looked up through ActiveSupport's cache store registry:

```ruby
IdentityCache.cache_backend = ActiveSupport::Cache.lookup_store(*Rails.configuration.identity_cache_store)
```

Going through ActiveSupport rather than talking to Memcached directly means the same process can be pointed at a different store in development or in tests, which is the kind of thing that makes a cache layer testable at all.

## Opting in at the model level

The library is opt-in per class, and the README is explicit that you must include the IdentityCache module into the classes where you want to use it.

The basic example shows a class that is cached without a primary index, and a class that is cached with one:

```ruby
class Image < ActiveRecord::Base
  include IdentityCache::WithoutPrimaryIndex
end

class Product < ActiveRecord::Base
  include IdentityCache

  has_many :images

  cache_has_many :images, embed: true
end
```

`WithoutPrimaryIndex` is the option for a model you never want to look up by its own identifier, which is a sensible thing for a child record that is always reached through its parent.

The second thing in that example is the first of the embedding options, `cache_has_many :images, embed: true`, and the comment above the fetch explains what it buys:

```ruby
# Fetch the product by its id using the primary index as well as the embedded images association.
@product = Product.fetch(id)

# Access the loaded images for the Product.
@images = @product.fetch_images
```

So `Product.fetch(id)` reads the primary index and the embedded images association from the same cache entry.

The `cache_has_one` counterpart takes the attribute to embed by, so `cache_has_one :featured_image, embed: :id` embeds the identifier rather than the whole object, and `fetch_featured_image` resolves it.

That distinction between embedding the whole object and embedding an identifier is the knob that controls how much data one cache entry carries, and it is the main thing to think about when you decide which associations to embed.

## Secondary indexes and the fetch methods they generate

IdentityCache lets you look up records by fields other than `id`, and you can have several indexes with any combination of fields:

```ruby
class Product < ActiveRecord::Base
  include IdentityCache
  cache_index :handle, unique: true
  cache_index :vendor, :product_type
end
```

Each index generates its own fetch methods. `Product.fetch_by_handle(handle)` looks up by the single-field index, `Product.fetch_multi_by_handle(handles)` takes an array of values, and the composite index produces `fetch_by_vendor_and_product_type(vendor, product_type)` and `fetch_multi_by_vendor_and_product_type` taking an array of pairs.

The `unique: true` option on the first index is worth pausing on. The documentation of `cache_index` says it allows you to say that an index is unique, only one object stored at the index, or not unique, which allows there to be multiple objects matching. A unique index stores one object; a non-unique index stores a collection, and the multi-fetch methods exist because of that second case.

The README is candid about the cost. This gives you a lot of freedom to use your objects the way you want to, and it does not get in your way, but it also keeps an independent cache copy in Memcached, so you might want to watch the number of different caches that are being added.

That is the structural trade of the whole library. Every index is another copy of the same data in the cache, and each copy has to be invalidated when the record changes. One index is cheap, ten indexes on the same table is ten copies, and the invalidation hook has to expire all ten.

## Embedding, polymorphic associations and caching a single attribute

Embedding is the feature that makes the cache worth having, and the README is precise about what it does and what it costs.

IdentityCache can easily embed objects into the parent's cache entry, so loading the parent also loads the association and adds it to the cache. Subsequent cache requests load the parent along with the association in one fetch. The warning follows: this can again mean some duplication in the cache if you want to be able to cache objects on their own as well, so it should be done with care. It works with both `cache_has_many` and `cache_has_one`.

The consequence for a cache miss is spelled out: the product and its associated images are loaded from the database, all of it is stored into the single cache key for the product, and later requests load the entire blob of data, so `fetch_images` does not need to hit the database because the images came with the product from the cache.

Polymorphic associations need help, because the other side of the association is not always determinable. IdentityCache tries to figure out both sides whenever it can so it can set them up when rebuilding the object from the cache, and in the hard cases you tell it what the association should be with `cache_belongs_to`:

```ruby
class Metafield < ActiveRecord::Base
  include IdentityCache
  belongs_to :owner, polymorphic: true
  cache_belongs_to :owner
end
```

And for the case where you do not need the whole object, `cache_attribute` caches a single attribute under a key you specify:

```ruby
class Redirect < ActiveRecord::Base
  cache_attribute :target, by: [:shop_id, :path]
end

Redirect.fetch_target_by_shop_id_and_path(shop_id, path)
```

That reads the attribute from the cache, or queries the database for the attribute and stores it in the cache, and it is the lightest-weight thing the library does, because a redirect's target changes rarely and is read constantly.

## The repository, and the caveats file you should read first

The top level of the repository shows a gem with the supporting material of a company that ships it.

There is `CAVEATS.md` at the root, and given what the library does that file is the first thing to read. There is a `performance/` directory, which for a caching library is the most interesting directory to look at, because the claims in the README are about read performance and the benchmarks behind them are usually kept out of the public tree. There is a `gemfiles/` directory for testing against multiple dependency sets, a `.spin/` directory, which is Shopify's development environment tool, a `dev.yml` describing it, and a `shipit.rubygems.yml`, which is the configuration for Shopify's internal deployment tooling. Those last three are a clear signal that this gem is released from inside a large engineering organisation rather than as a standalone project.

The build and quality tooling is conventional: a `Gemfile` and `Gemfile.lock`, a `Rakefile`, a `identity_cache.gemspec`, a `.rubocop.yml`, a `.ruby-version`, and a `CHANGELOG.md` and `CONTRIBUTING.md`.

The `lib/` directory holds the implementation, with the `fuzzy` subdirectory naming aside, and `test/` the test suite. The `CHANGELOG.md` and the release titles are the record of what has changed, and the most recent title, properly passing MemCacheStoreCAS serialized entries to Dalli, is a good illustration of what this library has to deal with: the difference between two memcached client implementations in how they serialise an entry for a compare-and-swap.

That is the unglamorous reality of a caching library. The interesting part is not the caching, it is the invalidation and the client compatibility.

## Where identity_cache is the wrong choice

The first wrong case is a read-once workload. If each record is read once and written once, a blob cache adds storage, adds invalidation traffic and adds a class of bug where the cache is stale, and returns nothing. The library is for hot, repeated reads of the same large objects, which is what Shopify's storefront does and what most applications do not.

The second is a codebase where `find` is the right mental model. IdentityCache is not a query cache. You cannot filter, sort or paginate through it, and the fetch methods are per-index lookups. An application whose reads are mostly listing and filtering wants a different tool.

The third is a team that cannot operate Memcached. The library assumes a shared cache cluster, and it assumes you understand the failure modes, which is why the configuration comments talk about failover and ejected hosts. A cache that silently splits into two views of the same key is a class of bug that is very hard to diagnose later, and the library's default of disabling failover is a recommendation born of having seen it go wrong.

The fourth is a schema with many secondary indexes. The README's own warning is that each index keeps an independent copy, and the invalidation has to expire all of them. A table with ten lookup keys is ten copies of every row.

The fifth is a team that needs read-your-writes across the whole system. The cache is invalidated on commit, so a read after a write on the same instance sees the new value, but a read on another instance depends on the invalidation message arriving. That is eventually consistent by design, and the six hour expiry is the backstop for when the message does not arrive.

For a large read-heavy Rails application with a Memcached cluster already in place, this is a mature, battle-tested implementation of a specific idea, and the fact that it came out of production rather than out of a design document is the argument for it.

## Conclusion

identity_cache suits a Rails application whose hot read paths are dominated by a small number of large, frequently read records, and that already runs Memcached and is willing to give up `find` in exchange for a cache hit. It does not suit a codebase where every model is read once and written once, because the cache entries duplicate data and the invalidation is the part that will be hard to reason about. Before adopting it, read `CAVEATS.md` in the repository root, and read the configuration comments about `expires_in` and about disabling failover, because those are the library stating the consistency trade-offs it is making rather than you.

## FAQ

### What does the identity_cache gem do?

It provides opt-in read-through ActiveRecord caching that stores whole model objects as blobs in Memcached, with secondary indexes, embedded associations and cache expiry driven by an after_commit hook that invalidates the changed object and any object up the tree that embedded it.

### How do I enable identity_cache in a Rails application?

Add the gem to your Gemfile, set `config.identity_cache_store` in your environment files with the cache store name, the memcached servers and the options, and add an initializer that calls `IdentityCache.cache_backend = ActiveSupport::Cache.lookup_store(*Rails.configuration.identity_cache_store)`.

### What is the difference between the dalli and memcached_store clients?

The `dalli` gem backs the `:mem_cache_store` option and the `memcached_store` gem is the old libmemcached based client. The installation section recommends Dalli, and the recent v1.6.5 release was about properly passing MemCacheStoreCAS serialized entries to Dalli.

### How do I look up a record by something other than its id?

Declare a secondary index with `cache_index`, optionally with `unique: true` for a single object or without it to allow multiple matches. Each index generates fetch methods such as `fetch_by_handle` and `fetch_multi_by_handle`, and composite indexes generate methods for the combination of fields.

### Why does the identity_cache configuration set expires_in and disable failover?

The comments in the configuration give the reasons. The expiry is there in case of network errors when sending a cache invalidation, so an entry cannot live forever, and failover is disabled because it avoids more cache consistency issues, since failing over to another memcached server can leave two views of the same key.

## Sources

- [License: MIT](https://github.com/Shopify/identity_cache/blob/main/LICENSE)
- [Project website](http://shopify.github.io/identity_cache)
- [README](https://github.com/Shopify/identity_cache/blob/main/README.md)
- [Releases](https://github.com/Shopify/identity_cache/releases)
- [Shopify/identity_cache on GitHub](https://github.com/Shopify/identity_cache)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/shopify-identity-cache