OpenLake: A Rust storage engine that keeps GPUs fed without code changes
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
At a glance
- What is it?
- OpenLake is a distributed storage engine built in Rust on io_uring, aimed at KV cache offload, checkpointing, and object storage for LLM training and inference. It plugs into vLLM with a connector and claims million-plus IOPS within 1ms, but you should verify its performance claims on your own hardware before adopting it.
- Who is it for?
- Adopt OpenLake if you run vLLM-based inference with long contexts and want to offload KV cache without changing model code, or if you need an S3-compatible object store with low-latency small I/O on GPU hosts. Do not adopt it if your workloads are mostly large sequential reads or if you cannot tolerate a project with only recent releases and performance claims that come from vendor blogs rather than independent benchmarks.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Rust, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What OpenLake actually solves
OpenLake targets a specific bottleneck: GPUs sitting idle while waiting for data. In LLM inference, the KV cache grows with context length, and for long prompts, prefill can dominate latency. OpenLake stores that KV cache on host RAM and disk, so a GPU can read it back in milliseconds instead of recomputing. In training, checkpointing is often a serialization point that stalls the cluster. OpenLake aims to make checkpoint writes and reads fast enough that GPUs do not wait. The project also lists VectorDB and context storage for agentic retrieval as use cases. The intended user is an engineer running vLLM or a training framework who wants storage that behaves like a cache but is persistent and durable. The README claims a 66x speedup on time to first token when cached for a 128K context window, and 8x throughput on a 100 TB KV cache scenario, but those numbers come from the project's own blog posts, not from a neutral benchmark you can reproduce from the README alone.
Architecture: io_uring, RDMA, and a client connector
The core engine is written in Rust and built on io_uring, the Linux asynchronous I/O interface. That choice is significant: io_uring allows high-throughput small I/O with low syscall overhead, which matches OpenLake's claim of million-plus IOPS within 1ms. The repository layout shows a server binary called openlaked and a set of crates, including openlake_server and openlake_client. The client side includes a vLLM connector module, openlake_client.openlake_connector, which implements vLLM's kv-transfer interface. That connector lets vLLM treat OpenLake as a KV cache store without modifying model code. For multi-host setups, OpenLake uses RDMA over InfiniBand or similar high-speed networking, as shown in the kv_rdma.toml config. The data flow is: a GPU node runs openlaked, the vLLM process connects to it via the connector, and KV data is written once and read back later, either on the same host or from a peer host. The README states that a prefix computed on one GPU host can be served from any other host in the shared pool, which implies a distributed namespace rather than a simple local cache.
Getting it running: two paths, one for vLLM, one for object storage
The quickstart shows two distinct deployment modes. For KV cache offload, you install a Python package and start a daemon: pip install openlake-vllm, then openlaked. After that, you launch vLLM with a --kv-transfer-config JSON that names the connector, the module path, and the OpenLake node addresses. A minimal config uses openlake_nodes like ["127.0.0.1:9400"] and openlake_device set to "local". For multi-host clusters, you start openlaked with a config file such as kv_rdma_0.toml on each node, giving each a self_id, and point vLLM at the list of node addresses. The second path is building an S3-compatible object store from source. That requires installing system dependencies like clang, cmake, libhwloc-dev, and libudev-dev, then running cargo build --release --bin openlaked. You start the store with a config like crates/openlake_server/configs/storage-tcp-local.toml, after creating data directories. Then you can talk to it with any S3 client by setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, both defaulting to openlakeadmin, and using an endpoint URL like http://127.0.0.1:90. Notice the README cuts off mid-command in that example, so the exact port and path are not fully shown.
The MLPerf claim and what it does not tell you
The most prominent claim in the README is that OpenLake tops MLPerf Storage v3.0 for object checkpointing, leading NVIDIA and Nebius. That is a strong statement, but the README only links to a blog post on the project's own site. There is no link to the official MLPerf results page, no benchmark methodology, and no raw numbers in the README. You cannot verify the claim from the supplied material. The blog post title says 'OpenLake leads MLPerf Storage v3.0', but leading a specific category in MLPerf Storage does not mean OpenLake is faster than all alternatives in every workload. MLPerf Storage measures specific benchmark scenarios, and object checkpointing is one narrow case. If you are considering OpenLake for general-purpose training storage, this claim should not be your deciding factor. You need to run your own benchmark with your own checkpoint sizes and I/O patterns.
A real limitation: the single-node default hides the distributed complexity
The README's quickstart for KV cache offload defaults to a single host: 'By default OpenLake offloads to the same host.' That is easy to set up, but it only helps if your GPU node has enough host RAM and disk to hold the KV cache. The whole point of offloading is to handle contexts that do not fit in GPU memory, and for very long contexts, a single host's RAM may still be insufficient. The multi-host RDMA setup is the scalable path, but it requires InfiniBand or similar hardware, as indicated by the openlake_device config value like mlx5_ib0, which is a Mellanox NIC device name. If your cluster uses Ethernet without RDMA, that path is not available. The README also mentions a Helm chart for Kubernetes, but it does not explain how OpenLake handles node failures or data replication. The storage engine is described as 'fully persistent and durable', but durability in a distributed system usually requires replication, and the README does not specify a replication factor or consistency model. That is a gap you should investigate before trusting it with critical checkpoints.
Alternatives: what else does KV offload and checkpoint storage?
The most direct alternative is vLLM's built-in KV cache manager, which can offload KV to CPU RAM or disk without a separate storage engine. That approach is simpler because it does not require an extra daemon or a network connector, but it is limited to a single node and does not provide a shared pool across hosts. Another alternative is a general-purpose distributed filesystem like JuiceFS or Alluxio, which can serve checkpoints and small files but is not designed specifically for KV cache access patterns. The key difference is that OpenLake is purpose-built for the GPU workload: it uses io_uring for low-latency small I/O and supports RDMA for fast peer-to-peer transfer, while a general filesystem typically has higher per-operation overhead and may not integrate with vLLM's kv-transfer interface out of the box. The README also mentions ExANS, a lossless GPU codec for BF16 KV cache that compresses data to save costs, which is a feature that generic storage engines do not offer.
Maintenance and license: what you are signing up for
OpenLake is licensed under Apache-2.0, which is permissive and allows commercial use without copyleft obligations, but you should read the license text yourself for any specific requirements. The project's release history shows v0.8.0 and v0.8.1 in August 2026, and v0.9.0 in September 2026, so it is under active development. The repository is not archived, and the last push was September 2026, a day after the v0.9.0 release. That activity is a positive sign, but it also means the project is young. Version numbers below 1.0 indicate that the API may change without notice. The README does not mention a migration guide or upgrade path between versions. If you adopt OpenLake, you should budget time for tracking releases and testing whether your configuration files and vLLM connector settings still work after an upgrade. The Rust codebase and the use of io_uring tie you to Linux kernels that support it, which is most modern kernels, but you should verify your kernel version.
Editorial conclusion
Adopt OpenLake if you run vLLM-based inference with long contexts and want to offload KV cache without changing model code, or if you need an S3-compatible object store with low-latency small I/O on GPU hosts. Do not adopt it if your workloads are mostly large sequential reads or if you cannot tolerate a project with only recent releases and performance claims that come from vendor blogs rather than independent benchmarks. Before committing, verify the MLPerf Storage v3.0 claim against your own hardware, test the vLLM connector with your exact model and context length, and confirm that the io_uring and RDMA requirements match your kernel and network stack. The project is Apache-2.0 licensed and under active development, but its long-term maintenance is unproven.
Community notes