# YCSB: a Java benchmark harness for comparing key-value and NoSQL stores

> YCSB ships a workload generator, a set of database bindings and a histogram-based latency reporter. It is for engineers who need repeatable load numbers from their own cluster, not vendor slides.

**brianfrankcooper/YCSB** — Yahoo! Cloud Serving Benchmark

- Repository: https://github.com/brianfrankcooper/YCSB
- Stars: 5,232 · Forks: 2,327
- Language: Java
- License: Apache-2.0
- Published: 2026-09-23 · Updated: 2026-09-23 · Language: en
- Canonical page: https://hysenlabs.com/projects/brianfrankcooper-ycsb

## What YCSB measures, and who needs that measurement

YCSB is a client-side load generator. It does not instrument the database; it opens connections through a binding, issues reads, writes, updates, inserts and scans according to a workload file, and records what the client observed. That design choice decides everything else about the tool. You get numbers that include network and driver overhead, which is what an application would feel, and you get nothing about compaction queues, cache hit ratios or replication lag inside the server.

The intended user is someone comparing two stores, two schema designs, or two cluster sizes under the same request mix. The repository carries bindings for a long list of systems, including cassandra, mongodb, redis, hbase1, hbase2, couchbase, dynamodb, elasticsearch, jdbc and rocksdb. Because the workload description is separate from the binding, the same workloada file can drive a MongoDB cluster and a Cassandra cluster, which is the whole point of a common benchmark.

The cost of that generality is abstraction. A binding implements the core operations, and anything the database does well but does not map onto read, insert, update, scan or delete simply does not appear in the results.

## The load and run phases, and why the split matters

A YCSB session has two distinct commands. The load phase populates the database with the record count named in the workload properties. The run phase executes the operation mix against that data and prints throughput and latency. The README's getting-started example uses the basic binding, which is an in-memory stub rather than a real database, so it is useful only to confirm the harness works.

The load and run split is not cosmetic. If you load and run in one pass you are measuring a database that is still absorbing writes, and its read latencies will reflect that. Running load to completion first gives you a steady state, but it also means your dataset size is fixed by the recordcount property and your working set may or may not fit in memory. The README does not document a rollback or cleanup command, so removing the loaded data is a task for the database's own tooling.

Workload properties are passed with -P, and the repository ships a workloads directory with the standard mixes. The README points to the wiki page on core properties for the full list, and to the running-a-workload page for the run mechanics.

## Latency percentiles, HDR histograms and the averaging trap

The README takes an unusually firm position on latency reporting. It states that the 99th percentile is the interesting figure and that the tail beyond it, 99.9%, 99.99% and 99.999%, matters more than the difference between P95 and P99. It gives the formula Probability_to_observe = 1 - Percentile ^ Requests, and works through the example that roughly 30% of users loading a default page with 30 requests will see something worse than P99.

The practical instruction that follows is the one worth remembering: latency percentiles cannot be averaged. If you run several loader processes, each produces its own distribution, and averaging their P99 values produces a number that describes nothing. YCSB supports this by writing raw histograms instead of summary statistics.

You enable that with two properties, hdrhistogram.fileoutput=true and hdrhistogram.output.path=file.hdr. The README then tells you to merge the files manually and extract percentiles from the joined result. It also warns that running multiple workloads at once may distort the distributions those workloads were designed to produce, which is a candid admission that parallel load generation changes what you are measuring.

## Installing YCSB and running a first workload

The README's getting-started path downloads the 0.17.0 tarball from the releases page, unpacks it and changes into the resulting directory. There is no package manager step and no installer; the tarball contains the bin scripts and the built bindings.

```bash
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
tar xfvz ycsb-0.17.0.tar.gz
cd ycsb-0.17.0
```

Before running against a real store, the README says to set up a database to benchmark and points to a README file under each binding directory. That per-binding file is where connection properties live, and it is the file to read first, because the top-level README does not document binding-specific keys.

The two commands below run against the basic in-memory binding with workloada. Running ycsb without arguments prints usage.

```bash
bin/ycsb.sh load basic -P workloads/workloada
bin/ycsb.sh run basic -P workloads/workloada
```

On Windows the equivalent scripts are bin\ycsb.bat with a backslash before workloada. To capture histograms instead of only summary output, add the two hdrhistogram properties to the run command.

```bash
bin/ycsb.sh run basic -P workloads/workloada -p hdrhistogram.fileoutput=true -p hdrhistogram.output.path=file.hdr
```

Building from source requires Maven 3; the README notes that Maven 2 can produce errors. A full build with every binding is mvn clean package, and a single binding is built with the module name, for example site.ycsb:mongodb-binding.

```bash
mvn clean package
mvn -pl site.ycsb:mongodb-binding -am clean package
```

## Where YCSB gives you a misleading answer

The most common failure is treating a basic-binding run as a result. It is a harness check, not a measurement of any database.

The second is ignoring the client side of the loop. YCSB runs on the machine you launch it from. If that machine is CPU-bound, network-limited or sharing a host with the database, the latency distribution you record belongs to your test rig as much as to the store. The README's own discussion of multiple loaders implies you can scale generation horizontally, but it also warns that concurrent workloads distort distributions, so adding loaders is not a free fix.

Third, the tool is the wrong instrument for questions it was never built to answer. It will not tell you how a database behaves under a schema migration, how it handles a hot partition, or what happens during a node failure. It measures client-observed operations against a synthetic key distribution. If your production access pattern is a graph traversal or an analytical join, the standard workloads will not resemble it, and writing a custom binding to fake it is more work than the comparison is worth.

Finally, the release history is a real constraint. The newest release listed is 0.17.0 from 2019-10-06, with 0.17.0-RC1 and 0.16.0 shortly before it. The repository's last push was 2026-08-12, so binding code has moved since the last tagged release. If you need a binding that was added or fixed after 0.17.0, you are building from source.

## YCSB against a driver-level microbenchmark

The obvious alternative is to write your own load generator against the database's native driver, or to use whatever benchmark the vendor ships. The difference is in what gets held constant. A hand-written generator gives you full control over the request shape and lets you measure an operation the binding does not expose, but every team that writes one makes different choices about key distribution, value size, thread count and warmup, so two such generators are rarely comparable.

YCSB's contribution is the fixed workload vocabulary. The workloads directory defines the standard mixes, and the core properties define record count, operation count, read proportion and the rest. Two teams running workloada with the same properties are running the same experiment, at least at the client level. That is worth more than a bespoke harness when the goal is to compare systems rather than to characterize one.

The trade-off is the reverse of what you might expect: the more you customize YCSB to match your production traffic, the less comparable your results become to anyone else's. Use the stock workloads when you need a common reference point, and a custom generator when you need fidelity to a specific application.

## Conclusion

Adopt YCSB if you control the database deployment and need a repeatable load generator whose workloads you can read and edit. Do not adopt it if you need a managed service's internal metrics, or if you expect a maintained release cadence: the newest release is 0.17.0 from 2019-10-06, while the repository's last push was 2026-08-12. Before trusting any number, verify that your binding's README exists under its directory and that you have captured hdrhistogram.fileoutput so percentiles survive multiple loaders.

## FAQ

### How do I install YCSB?

Download the latest release tarball, unpack it and change into the directory, as the README's getting-started section shows with version 0.17.0. Building from source instead requires Maven 3.

### What is the difference between the load and run commands in YCSB?

The load phase populates the database with the record count from the workload properties, and the run phase executes the operation mix against that data. The README's example uses the basic binding for both.

### Why does YCSB report latency percentiles instead of an average?

The README states that latency percentiles cannot be averaged and that neither latency averages nor P99 averages make sense. It points to the tail beyond P99 as the figure of interest.

### How do I merge YCSB results from multiple loaders?

Run the loaders with hdrhistogram.fileoutput=true and hdrhistogram.output.path=file.hdr, then merge the resulting HDR files and extract percentiles from the joined result. The README suggests the HdrLogProcessing CLI for the union and summarize operations.

## Sources

- [brianfrankcooper/YCSB on GitHub](https://github.com/brianfrankcooper/YCSB)
- [Issues](https://github.com/brianfrankcooper/YCSB/issues)
- [License: Apache-2.0](https://github.com/brianfrankcooper/YCSB/blob/master/LICENSE)
- [README](https://github.com/brianfrankcooper/YCSB/blob/master/README.md)
- [Releases](https://github.com/brianfrankcooper/YCSB/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/brianfrankcooper-ycsb