Curvine: a POSIX file layer over cloud object storage, with a tiered cache

The gap Curvine fills: object storage durability without the object API tax

Object storage is cheap, durable and multi-region, but it is a poor fit for workloads that expect a file system. The README frames Curvine as a high-performance POSIX file semantic layer built on top of cloud object storage, with an integrated multi-tier distributed cache, designed for large-scale AI workloads and AI Agent platforms. The concrete pain is the round trip: every open, read, write, seek, rename and list becomes an HTTP call against a bucket, and the latency of that call sets the floor for everything above it.

Curvine's answer is to keep object storage as the durable layer and put a distributed file system in front of it. The README states that Curvine exposes full POSIX semantics upward while using object storage as the durable persistence layer downward. The intended audience is narrow and specific: the README names AI Agent Pods (10,000+ stateful workloads), AI and big-data engines (training, inference, OLAP), and the native cv CLI as the workloads that use it. If you run a small number of large sequential readers, the object API is not your problem and this layer adds moving parts you do not need.

Control plane, data plane, and where the bytes actually go

Curvine splits into a Control Plane and a Data Plane. The Master node is Raft-replicated and manages metadata, namespace, scheduling, load balancing and cluster coordination, with a Web UI and API alongside it for dashboarding, metrics and management. Worker nodes serve data from a multi-tier cache (Memory to SSD to HDD) with automatic hot-data promotion, eviction and replication. The README describes the data flow plainly: metadata operations are routed to the Master via RPC, while data I/O is served directly by the Workers. On a cache miss, Workers fetch from and persist back to the underlying object storage.

The design decision worth noting is metadata independence. The README says Curvine's file metadata path maps 1:1 to the underlying S3 object path, so even if the Curvine service is unavailable, objects on S3 keep their original structure and remain independently accessible. That is a real escape hatch, and it is also a constraint: any layout that cannot be expressed as a 1:1 mapping to object paths is out of scope. The storage layer covers AWS S3, Azure Blob, Google GCS, OSS, and any S3-compatible store such as MinIO or HDFS, and the workspace Cargo.toml lists adapter crates including curvine-ufs-opendal, curvine-ufs-oss-hdfs and curvine-hdfs-jni, which is where that multi-backend support is implemented.

Access paths are deliberately plural. The README lists POSIX FUSE (curvine-fuse), an S3-compatible gateway, an HDFS/UFS adapter, Java/Python/Rust SDKs, and a native Kubernetes CSI driver. The point of the FUSE layer is that existing tools work unmodified: the README names Vite, inotify/fswatch and git as examples that run against it without changes.

Installing Curvine and mounting a first PVC

The README does not carry install steps inline. It points to the Quick Start at https://curvineio.github.io/docs/Deploy/quick-start and describes Helm-based cluster deployment as a feature. What the repository itself shows is a Makefile-driven build with Docker targets, so the source path is the one you can verify from the tree.

The Makefile exposes a help target that lists the available commands, including environment checks and the build entry points. Run it first to see what your machine is expected to provide:

make help

The output groups commands under Environment, Building and Docker. Environment includes make check-env, make check-client-deps, make check-api-crate-deps and make check-minimal-artifact-deps. Building includes make build ARGS='<args>', make all, make dist, make dist-only, make format and make format-csi. Docker includes make docker-build, which the help text describes as building a runtime Docker image from source.

To produce a distribution package, the Makefile documents make dist as build and create distribution package (tar.gz), and make dist-only as creating the package without building:

make dist

For Kubernetes, the README states that the native CSI driver mounts the FUSE file system directly as a PVC, with Immediate binding and volume expansion, so provisioning is a mkdir on the shared namespace rather than a cloud control-plane API call. The repository contains a curvine-csi directory and a make format-csi target for its Go code, which confirms the CSI driver is a separate Go component rather than part of the Rust workspace. The README does not document the Helm chart values, so treat the Quick Start page as the authority for chart configuration.

Where Curvine is the wrong tool

The README's performance claims are stated without the conditions that produce them: a Rust core with Tokio async runtime, zero-copy data paths and a GC-free memory model, described as ~100μs-class latency and 100K+ stable QPS. Nothing in the README says on what hardware, with what cache hit ratio, or against which object store. A 100μs-class figure only makes sense on a cache hit; a miss still pays the object storage round trip, and the README does not publish miss-path numbers. If your access pattern is cold-read dominated, the cache tier is not doing the work and Curvine is an extra hop.

The second limitation is operational surface. A working cluster means a Raft-replicated Master, a Worker fleet, a CSI driver, and a FUSE mount on every consuming node. The README does not document a single-node or embedded mode, and the workspace layout (curvine-master, curvine-worker, curvine-server, curvine-mds as separate members) reflects that. Teams without Kubernetes operators or a storage on-call rotation should weigh that honestly.

The third is the metadata mapping. The 1:1 metadata-to-S3-path property is presented as an advantage for recovery, and it is, but it also means Curvine is not a general namespace layer. If you need a virtual namespace that diverges from object paths, this design works against you.

Finally, the project is young. The most recent release listed is v0.4.1 from 2026-08-11, preceded by v0.4.0-alpha and v0.3.6-alpha. The last push to the repository was on 2026-08-11. Alpha-tagged releases in the recent history are a signal about interface stability, not about the maintainers' effort.

How Curvine differs from JuiceFS and Alluxio

JuiceFS is the closest comparison and the difference is in where metadata lives. JuiceFS keeps file metadata in a separate transactional database (Redis, TiKV, MySQL and similar) while data chunks go to object storage. Curvine instead maps its metadata path 1:1 onto the underlying S3 object path, and replicates the Master's metadata with Raft rather than delegating to an external database. That removes a database from your operational stack and gives you the property that objects remain readable and correctly structured on S3 if Curvine itself is down. The cost is that you cannot reshape the namespace independently of the bucket layout.

Alluxio approaches the same problem from the data-orchestration side, presenting a virtual namespace across multiple under-stores with a strong emphasis on caching and tiering. Curvine's README positions it as a file system rather than a federation layer: POSIX semantics via FUSE, an S3-compatible gateway, an HDFS/UFS adapter, and SDKs. If your need is to unify several existing storage systems under one namespace, Alluxio's model matches that goal more directly. If your need is a POSIX mount over one object store with a local cache, Curvine's model is the tighter fit.

Neither comparison is settled by the README alone. Curvine's own benchmark page is at https://curvineio.github.io/docs/category/benchmark, and the README links an AWS machine learning blog post about tiered KV cache for large LLMs on Amazon SageMaker HyperPod with Curvine, which is the closest thing to an independent write-up linked from the project.

Licence, upgrade cost and what the tree tells you about maintenance

Curvine is Apache-2.0, and the repository carries the LICENSE file plus a DCO file, a CODE_OF_CONDUCT.md, CONTRIBUTING.md, COMMIT_CONVENTION.md, MAINTAINERS.md and ADOPTERS.md. The Apache-2.0 grant includes an explicit patent grant, which matters if you are embedding the client SDKs in a product rather than only mounting the file system. That is a description of the licence text, not legal advice; have counsel review anything you ship.

Upgrade cost is shaped by the release cadence visible in the repository. v0.3.6-alpha, v0.4.0-alpha and v0.4.1 land within roughly a month of each other, and two of the three carry an alpha tag. The README does not document an upgrade procedure, a compatibility policy between Master and Worker versions, or a rollback path. In a Raft-replicated system, the Master/Worker protocol version is the thing you would want pinned before an upgrade, and the README is silent on it. Plan to test upgrades in a non-production cluster and to read the release notes for each tag rather than assuming drop-in compatibility.

The repository is not archived, and the last push was on 2026-08-11. The project describes itself as a CNCF Sandbox Project, and the README links to its entry in the CNCF landscape. Sandbox status is a stage, not a durability guarantee. The presence of a SECURITY.md, a .gitvote.yml and a devcontainer setup suggests a project that has thought about contributor process; that is process, not production track record.

Editorial conclusion

Adopt Curvine if you run thousands of stateful Agent pods or training jobs on Kubernetes against S3-compatible object storage and the object API round trip is what limits you. Do not adopt it if your data already sits on a parallel file system, or if you cannot operate a Raft-replicated Master plus a Worker fleet: the README does not document a single-binary mode. Before committing, verify three things in your own environment: that curvine-csi binds PVCs the way your cluster expects, that the FUSE layer passes your tooling (the project's LTP result is a compatibility signal, not a substitute for your own test), and that the 1:1 metadata-to-S3-path mapping holds for the bucket layout you already have, since that mapping is what makes recovery without Curvine possible.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes