S3Proxy: One S3 API In Front of Files, Clouds, and SFTP

What S3Proxy Actually Solves

S3Proxy is a Java service that implements the S3 API and proxies requests to another storage backend. The README lists three use cases: translating S3 calls to Google Cloud, Azure, OpenStack Swift, and SFTP; testing without Amazon by using the local filesystem; and extending behavior via middlewares. The target audience is developers who have tooling written against the S3 API, but who do not want to run against real AWS or who need to point that tooling at a different vendor. It is not a storage system itself. It is a translation layer that makes a filesystem directory look like a bucket, or an Azure container look like an S3 bucket, to any client that speaks S3.

How the Proxy Mechanism Works

S3Proxy sits between an S3 client and a backend blobstore. It accepts S3 REST requests on an endpoint, then uses the jclouds library to translate them into backend-specific operations. The README shows that the configuration is driven by a properties file with two key groups: s3proxy.* for the proxy's own behavior, and jclouds.* for the backend connection. For example, jclouds.provider=filesystem tells it to use the local disk, while jclouds.provider=aws-s3 points it at real AWS. The proxy handles authentication, bucket mapping, and request signing. It also supports a credential supplier mechanism: when you embed S3Proxy in a Java application, you can hand BlobStores.create a Supplier<Credentials>, and the store will ask that supplier for every request it signs. This matters for rotating credentials like AWS session tokens or Keystone tokens, because the store does not cache the first answer.

Getting It Running with a Properties File

The quickest path is Docker, but without Docker you download a release or build from source with mvn package, which produces target/s3proxy. The project requires Java 17 or newer. You then write a properties file. The README gives a minimal example for the filesystem backend with anonymous access: s3proxy.authorization=none, s3proxy.endpoint=http://127.0.0.1:8080, jclouds.provider=filesystem, and jclouds.filesystem.basedir=/tmp/s3proxy. You must create the basedir first. On Linux or Mac you chmod +x s3proxy and run s3proxy --properties s3proxy.conf. On Windows you run java -jar s3proxy --properties s3proxy.conf. A curl PUT to /testbucket creates a bucket, and a GET to / lists buckets. That is the entire setup for a local test endpoint.

Credential Handling: What Rotates and What Does Not

The README goes into unusual detail about credentials, and that detail reveals a real design constraint. jclouds.identity and jclouds.credential name the enduring half of a credential. jclouds.session-token names the expiring part: an AWS session token, an Azure shared access signature, a Google OAuth 2.0 access token, or a Keystone token. A token written into the properties file lives no longer than the token itself. For rotating tokens, the recommended path is to embed S3Proxy and supply a Supplier<Credentials> that the store re-queries per request. However, three credentials are read only when the store is built: an Azure account key, a Google service account key, and the SFTP password. That means if you use those static credentials, a rotation requires rebuilding the store, which in practice means restarting the proxy. This is a limitation to plan around, not a bug.

Bucket Assignment and Middlewares

You can map different buckets to different backends in one proxy instance. The properties file uses s3proxy.bucket-locator.1=bucket and s3proxy.bucket-locator.2=another-bucket. Glob syntax is supported, so a single entry can cover many bucket names. A bucket or glob cannot be assigned to multiple backends. Beyond that, middlewares modify behavior. The README lists twelve, including bucket aliasing, prefix scoping, eventual consistency modeling, latency injection, read-only mode, regex rename blobs, sharded backend containers, and storage class override. These middlewares are what make S3Proxy more than a dumb translator. They let you simulate S3's eventual consistency for testing, or force a read-only view of a backend that would otherwise accept writes. That is a concrete mechanism for test scenarios that would otherwise require a mock service.

Where It Falls Short: The Unsupported S3 Features

The README is explicit about what S3Proxy does not support. The list includes ACLs other than private and public-read, BitTorrent, bucket inventory, analytics, metrics, lifecycle configuration, logging, notification, policies, replication, and CORS bucket operations. That is a significant portion of the S3 API that many applications take for granted. If your client code sets a bucket policy or configures lifecycle rules, it will fail against S3Proxy. The proxy is a good fit for basic object read and write patterns, but it is the wrong tool for workloads that depend on governance or automation features. The README also notes that the list is truncated in the source material, so there may be more unsupported operations than those named here. You should check the full wiki before committing.

Comparing to a Direct SDK or a Full S3 Emulator

The main alternative is to use the backend's native SDK or API instead of S3. If your application only targets Azure, you could write directly against Azure Blob SDK and skip the proxy layer. That avoids the translation overhead and the feature gaps. But it locks you into one vendor. S3Proxy's value is that you keep S3 as the common interface and swap backends underneath. Another alternative is a full S3 emulator like MinIO, which implements the S3 API natively and supports many more S3 features, including bucket policies and lifecycle. MinIO is a storage server, not a proxy, so it does not translate to Azure or Google Cloud. The difference is approach: S3Proxy translates to existing backends, while MinIO replaces the backend with its own storage. If you need to test against a realistic S3 implementation, MinIO is closer to AWS. If you need to expose an existing Azure or filesystem store via S3, S3Proxy is the more direct fit.

Maintenance, Licensing, and Upgrade Cost

S3Proxy is licensed under Apache-2.0, which permits commercial use, modification, and redistribution with attribution. The project is actively maintained: the latest release, 4.1.0, was pushed in August 2026, and 4.0.0 and 3.3.0 came in the months before. That cadence suggests regular maintenance. The upgrade cost depends on how you run it. Docker users can pull a new image. Kubernetes users have reference manifests in examples/kubernetes, including health probes and Secret-based credentials, but those manifests are examples, not a managed operator. Java embedders must rebuild and redeploy their application. The credential supplier mechanism is the main API surface to watch: if you rely on it, a version change could alter the contract. The README does not document a migration guide for version jumps, so you should read the release notes for each upgrade.

Editorial conclusion

Adopt S3Proxy if you need a local S3-compatible endpoint for testing, or if you must expose a non-S3 backend like Azure, Google Cloud, Swift, or SFTP through the S3 API. Skip it if you require full S3 feature parity, especially ACLs beyond public-read, bucket policies, lifecycle rules, or notification configuration, none of which are supported. Before committing, verify that your workload does not depend on the listed unsupported operations, and confirm that your backend's credential rotation model matches what S3Proxy reads: session tokens work, but Azure account keys and Google service account keys are read only at store build time, so those cannot rotate without a restart.

Official sources

1. Official README
2. Project repository
3. Release notes