S3 Ninja emulates the Amazon S3 API so you can develop against it locally

A local S3, not an S3

S3 Ninja is described as emulating the Amazon S3 API for development and test purposes, and that framing sets the expectation correctly. The project comes from Scireum, it is written in Java, it is MIT licensed, and the site s3ninja.net carries the fuller documentation. The design goal is narrow and valuable: you want your application, and its test suite, to talk to something that behaves like S3 without talking to AWS. That means no credentials in your test environment, no network dependency, no per-request cost and no risk of a test run deleting something in a real bucket. The obvious limit follows directly from the word emulate. An emulator implements the parts of an API that a development workflow touches, and a codebase that uses S3 features at the edge, or relies on exact consistency and durability semantics, will find gaps. The repository layout is conventional for a Java service on the Sirius platform, with a pom.xml, a src/ directory, a Dockerfile, a docker-compose.yml, a develop.conf and a .drone.yml for CI, and the default branch is develop rather than master.

Bringing it up with Docker Compose

The quick start assumes Docker and Docker Compose are installed, and from the project root the whole thing is one command.

   docker-compose up --build

That builds the image and starts the service on port 9444, with data persisted in the ./data directory. The compose file in the repository shows the shape of the deployment. The service is named s3ninja with the container name s3ninja-local, the port mapping is 9444 on the host to 9000 in the container, and there are two volumes: a bind mount of ./multipart for upload staging and a named volume for logs. The environment sets S3NINJA_PORT to 9000 and gives the JVM an explicit heap, with JAVA_XMX at 1g and JAVA_OPTS setting a maximum and initial heap along with a trust store pointing at the system CA certificates. A healthcheck polls the UI endpoint every 30 seconds with a 10 second timeout, three retries and a 40 second start period, which matters because the service takes a moment to become ready.

Building and running the image by hand

The manual path is documented for when you are iterating on the code rather than just using it. The image build requires Maven artifacts in target/, so the build is a two step affair.

  docker build -t s3ninja:local .

And the container run maps the host port to the container port and persists the data directory.

  docker run --rm -p 9444:9000 -v %cd%/data:/data s3ninja:local

After that there are two addresses to know, the browser UI at http://localhost:9444/ui and the S3 API endpoint at http://localhost:9444/. The development workflow section adds the practical details. Mount the local data/ directory to persist buckets and objects across container restarts, and for code changes rebuild the image after running mvn clean package -DskipTests, which is why the build step expects artifacts to already exist in target/. The Dockerfile itself is thin, based on the scireum/sirius-runtime-jre26 image, creating the data, multipart and logs directories, adding the pre-built release directory with the right ownership, switching to the sirius user, declaring the data and logs volumes and exposing port 9000.

The presigned POST workbench, and why it is the interesting part

The feature that goes beyond bucket CRUD is support for AWS S3 compatible presigned POST requests, and the documentation treats the debugging angle as the point. The web UI has a Presigned POST Workbench, reachable at http://localhost:9444/ui/presigned-post, where you fill in bucket, key and region, optionally generate a session token inline, and hit Generate Presigned POST. The results card shows the policy JSON, the Base64 policy and the signature, which is exactly what you need when a direct upload is being rejected and you need to know which condition in the policy is wrong. There is an embedded Test Upload panel so you can POST a file against your running instance without any Maven work. The request itself goes to a bucket and key path with a multipart/form-data content type, and the form fields are the generated fields plus the file. The example request in the documentation shows the full set of headers a browser would send.

curl -X POST "http://localhost:9444/test/example.txt" \
  -F "key=example.txt" \
  -F "policy=<BASE64_POLICY>" \
  -F "x-amz-algorithm=AWS4-HMAC-SHA256" \
  -F "x-amz-credential=<ACCESS_KEY>/<DATE>/<REGION>/s3/aws4_request" \
  -F "x-amz-date=<YYYYMMDDTHHMMSSZ>" \
  -F "x-amz-signature=<HEX_SIGNATURE>" \
  -F "[email protected]"

If you are using a session token from the /session-token endpoint, you add one more field.

-F "x-amz-security-token=<SESSION_TOKEN>"

The generated policy enforces conditions such as bucket, key, expiration and an optional security token, and on success the file is uploaded, while on failure an XML error comes back.

What the emulator does not attempt to be

Reading the repository carefully, the boundaries are visible. The Dockerfile exposes a single port and mounts three directories, and there is no configuration surface for anything resembling accounts, policies, versioning or lifecycle rules, which tells you the service is not modelling the S3 permission or bucket configuration model. The healthcheck polls the UI rather than the API, which is a sensible choice for a service whose contract is the API and whose UI is a convenience. The compose file sets a heap ceiling of 1g, and the presence of a separate multipart directory and a logs volume shows that upload staging and request logging are first class concerns for a service whose purpose is to make uploads work in development. The service also exposes a session token endpoint, which is there to support the STS style credential shape that presigned POST requires, and again this is the minimum needed to make that one flow work rather than an attempt at the wider STS API. So the honest characterisation is a functional subset aimed at the paths an application exercises, with a UI that shows you what it stored.

Where S3 Ninja is the wrong tool

The limits are structural and they are the same ones you would expect from a development-only emulator. The first is fidelity. If your application depends on S3 behaviour that the emulator does not reproduce, whether that is eventual consistency, object versioning, lifecycle expiry, bucket policies or replication, then a test that passes against S3 Ninja tells you little about production. The second is the trust boundary. A developer with a running instance on port 9444 can create and delete buckets and objects, and the persistence is a directory on your disk, so the instance has to be treated as a development tool on a developer machine rather than something you expose on a shared network. The third is that it is a service with a JVM, a database and a search server behind it, which is a heavy dependency for a unit test suite, so many teams will use it for integration tests and keep unit tests on an in-memory fake. The fourth is the platform coverage, since the Docker image is built for Linux containers and the repository's CI runs on Drone, so running it natively on a developer workstation is a different path with its own setup. And the fifth is the alternative itself, because a good in-process fake of the handful of S3 calls your code makes is often faster and easier to assert against than a running server, and this tool is for the cases where the surface is too wide to fake but too risky to point at AWS.

Editorial conclusion

Adopt S3 Ninja if your application writes to S3 and you want a local, offline stand-in so integration tests do not need AWS credentials or network access, since it speaks the S3 API and comes with a UI for poking at buckets directly. Do not adopt it as a production object store, since it exists to be a development and test target, and do not adopt it if your code depends on S3 features the emulator does not implement, because the project describes itself as an emulator rather than a complete implementation. Verify two things before you wire your test suite to it: that the specific S3 operations your code performs are covered, and that the presigned POST flow works in your setup, since that is a separate code path with its own policy generation. The practical first step is to bring the container up with docker-compose up --build, open the UI at http://localhost:9444/ui, and create a bucket before pointing any test at it.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. scireum/s3ninja on GitHub