soci-snapshotter: lazy loading OCI images without converting them

Image download is 76 percent of startup, and most of it is unused

The README frames the problem with a measurement rather than an opinion. The standard way to launch a container begins with a setup phase in which the image data is completely downloaded from a remote registry and a filesystem is assembled, and the application does not start until that finishes.

Citing Harter et al from FAST '16, the README states that image download accounts for 76% of container startup time, while on average only 6.4% of the fetched data is actually needed for the container to start doing useful work. That gap is the entire motivation. A workload that reads a handful of files out of a multi-gigabyte image still pays for the whole download before the process starts.

The approach SOCI takes is to remove the requirement to download the whole image before launch, to load data on demand instead, and to prefetch data in the background. Lazy loading handles the first access; the prefetch path is what keeps the second access fast.

The project also publishes benchmarks of its own, linked from the README as Website Benchmarks, hosted at awslabs.github.io. Those are the numbers to look at when deciding whether the mechanism fits your image sizes, since the 76% figure comes from 2016 research rather than from a measurement of your own fleet.

The SOCI index sits beside the image instead of replacing it

Existing lazy loading snapshotters depend on a build-time conversion step that produces a new image artifact. The README lists why that is a problem for container developers: it does not suit teams that will not or cannot modify their CI/CD pipeline, it does not suit teams that do not want the cost and complexity of keeping images in two formats, and it breaks image signing, because the conversion invalidates any signature created against the original OCI image.

SOCI loads from the original, unmodified OCI image. Rather than converting the image, it builds a separate index artifact, the SOCI index, which lives in the remote registry right next to the image. At container launch time, the snapshotter queries the registry for the presence of that index using the mechanism developed by the OCI Reference Types working group.

The security argument is the one that tends to convince teams. Because the bytes that run are the bytes that were signed, signing keeps working without a second signing pipeline for a derived artifact. The cost moves to the registry side, where an extra artifact has to be published, and to the launch path, where a registry query is added.

The repository layout matches this structure. There are dedicated `soci/` and `ztoc/` directories, with the latter holding FlatBuffers schema files for `ztoc.fbs` and `zinfo.fbs` and the generated Go code, plus `snapshot/`, `fs/`, `cache/`, `metadata/`, `service/`, `cmd/` and `config/` directories. The `go.mod` also lists `github.com/google/flatbuffers v25.12.19+incompatible` and `oras.land/oras-go/v2 v2.6.2` for the index artifact handling.

Index Manifest v1 avoids conversion, and v2 puts it back on purpose

The most interesting paragraph in the README is a note attached to the No image conversion section, and it is unusual for a project to argue against its own headline feature so explicitly.

The note says the section describes SOCI Index Manifest v1, and that while the absence of an image conversion step is appealing, for production scenarios it also creates the potential for performance changes across multiple dimensions if an index is added to or removed from an existing image that is widely deployed. To address that, the project introduced SOCI Index Manifest v2, which does use a build-time conversion step.

Read carefully, that means the no-conversion promise is versioned, not absolute. In v1, the behaviour of an image depends on whether a SOCI index happens to exist for it in the registry, so adding or removing that index changes how that image loads. For a fleet pulling the same widely deployed image from many nodes, that is a change in performance characteristics applied to a live workload, which is exactly the kind of surprise the note warns about.

v2 gives that up. The index becomes part of the image's own identity at build time, so the load behaviour is fixed when the image is produced rather than varying with registry state. The trade-off is that you are back to a build pipeline step, which is what the project originally set out to remove.

The README points to `docs/soci-index-manifest-v2.md` for the details. If you are evaluating this project, that document, not the overview, is the file that decides whether it fits your pipeline.

Installed as a systemd unit, with prefetch still marked experimental

The documentation list in the README describes a set of operational surfaces, and the wording tells you which ones are settled. There is a Getting Started guide for walking through setups and features, a Build document for building from source, testing and contributing, an Install document for installing SOCI as a systemd unit, and a Debug document for logs, metrics and common errors.

The systemd unit is visible in the repository root as `soci-snapshotter.service` and `soci-snapshotter.socket`, which is how the snapshotter is wired into a containerd host. The presence of a socket unit in the root directory tells you it is designed to be a long running service on the node rather than a one-shot command.

One capability is explicitly provisional. The Prefetch document is described as understanding and managing prefetch artifacts, and it is marked experimental. That matters because prefetching is the mechanism that turns lazy loading from a latency win on the first read into a benefit across the life of a pod. If you plan to depend on background prefetch, you are depending on the experimental part of the project first.

There is also integration documentation for two specific environments: an overview of running SOCI on Kubernetes, and a walkthrough for setting it up on Amazon EKS. Alongside them, the `go.mod` pulls in the Kubernetes libraries at v0.34.10, including `k8s.io/cri-api`, which is the interface a container runtime such as containerd exposes to Kubernetes.

Building from source pins containerd 1.7.32 and a Go 1.26 module

The Build document in the README covers building SOCI from source, testing it and contributing. The repository's own files show what that build environment looks like.

The Dockerfile pins the toolchain versions used for the integration environment.

ARG CONTAINERD_VERSION=1.7.32
ARG RUNC_VERSION=1.3.3
ARG NERDCTL_VERSION=2.1.6
ARG CRICTL_VERSION=1.36.0
ARG IGZIP_VERSION=2.31.1
ARG RAPIDGZIP_VERSION=0.16.0

The `igzip` and `rapidgzip` stages exist because the plugin needs fast decompression for the compressed layers it serves lazily, and the Dockerfile builds ISA-L from source for that reason. A registry image is also pulled in as a build stage, which is the local registry used by the integration tests rather than anything a production install needs.

The Go module itself is minimal in structure and pinned to a recent language version.

module github.com/awslabs/soci-snapshotter

go 1.26.0

The Makefile provides the usual knobs for where artifacts land, with `OUTDIR` defaulting to an `out` directory inside the project and `CMD_DESTDIR` defaulting to `/usr/local` for installation. It also derives `VERSION` from `git describe` and `REVISION` from the current commit, which is how a release build gets its version stamp, and it wires in the Apache license header template from the `.headers/` directory.

The `go.mod` depends on `github.com/containerd/containerd/v2 v2.2.9`, so the containerd API surface you compile against is explicit and can be checked against the containerd version running on your nodes.

A stargz fork that wants to be a containerd non-core project

The Project Origin section explains the lineage plainly. This project began as a fork of the popular stargz-snapshotter project from a specific upstream commit, with the intention of submitting an upstream patch. During development the changes were fundamental enough that the decision was made to create soci-snapshotter as a standalone project. The README says the project builds on stargz's success and innovative ideas, and that long term it intends and hopes to join containerd as a non-core project while following CNCF best practices.

The alternative worth understanding is stargz itself, since the difference is about the artifact rather than the runtime. stargz produces an eStargz image, a converted image format that carries an index of compressed chunks inside the image itself, and the runtime pulls chunks from that image. SOCI keeps the original image untouched and stores the equivalent index as a separate artifact in the registry, discovered through the OCI reference types mechanism. One produces a second format you must distribute and sign, the other produces a second object you must publish and keep in sync with the image.

Both are lazy loading snapshotters in the containerd community, and the README acknowledges there are several such projects. The differentiator here is the no-conversion property, and as the note in the design section explains, that property is what v2 partially gives back.

The project also states an intention about governance rather than a fact about today, which is worth separating. Joining containerd as a non-core project is a stated hope, not an accomplished status.

Workload-specific load order is the missing piece, and releases move fast

Under Design considerations, the README names what the project has not done. Workload-specific load order, meaning choosing the order in which image data is fetched based on the particular workload rather than a generic order, is described as a big consideration that has not been implemented or integrated into SOCI, with a pointer to the design README for more detail.

This is the honest boundary of the project. Lazy loading removes the need to fetch everything, but the ordering question determines how much of the remaining data you fetch before the container becomes useful. Without workload specific ordering, the snapshotter is working from a generic assumption about what a container touches first, and that assumption may not match your service.

The project is moving quickly. The three most recent tags are v0.16.1 on 2026-09-30, v0.16.0 on 2026-09-21 and v0.15.0 on 2026-07-30, and the last push was on 2026-10-20. The repository is not archived, and a `RELEASES.md` sits at the root next to `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md` and `NOTICE.md`. The `go.mod` targets `go 1.26.0` and depends on `github.com/containerd/containerd/v2 v2.2.9`, with `github.com/hanwen/go-fuse/v2 v2.11.0` for the FUSE layer, `github.com/klauspost/compress` for decompression and Prometheus client libraries for the metrics the Debug document refers to.

Licensing is Apache-2.0, with the `LICENSE` and `NOTICE.md` files at the root and Apache headers enforced through the `.headers/` directory and the `LTAG_TEMPLATE_FLAG` in the Makefile. Apache-2.0 carries a patent grant and an explicit notice requirement, which is a heavier compliance footprint than MIT, and it suits an infrastructure component in a larger organisation with a standard policy.

Editorial conclusion

Adopt soci-snapshotter when you run containerd based fleets that pull large standard OCI images and where the image download dominates startup, and when you can tolerate adding an index artifact next to each image in your registry. Do not adopt it if your images are converted by a build pipeline already, or if the risk of an index being added to or removed from a widely deployed image is unacceptable in your environment. Verify first that your containerd version and the SOCI snapshotter systemd unit work on the nodes you control, and read the SOCI Index Manifest v2 document before you commit, since v1 and v2 differ on whether a build-time conversion step exists.

Frequently asked questions

Official sources

1. awslabs/soci-snapshotter on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases