BenchmarkDotNet: turning a method into a measurement you can defend

Writing a benchmark looks like writing a test

The pitch in the README is that BenchmarkDotNet is no harder than writing unit tests, and the example proves it. You mark a class with attributes describing which runtimes to try, mark fields with the values to parameterize over, add a setup method, and mark the methods under test.

[SimpleJob(RuntimeMoniker.Net472, baseline: true)]
[SimpleJob(RuntimeMoniker.NetCoreApp30)]
[SimpleJob(RuntimeMoniker.NativeAot70)]
[SimpleJob(RuntimeMoniker.Mono)]
[RPlotExporter]
public class Md5VsSha256
{
    private SHA256 sha256 = SHA256.Create();
    private MD5 md5 = MD5.Create();
    private byte[] data;

    [Params(1000, 10000)]
    public int N;

    [GlobalSetup]
    public void Setup()
    {
        data = new byte[N];
        new Random(42).NextBytes(data);
    }

    [Benchmark]
    public byte[] Sha256() => sha256.ComputeHash(data);

    [Benchmark]
    public byte[] Md5() => md5.ComputeHash(data);
}

Two details matter more than they look. `[Params(1000, 10000)]` makes the framework run the whole benchmark once per value with N set accordingly, and `[GlobalSetup]` runs once per parameter set rather than per iteration. Getting the second one wrong is the classic way to measure your own allocation code instead of your hash function, and the framework warns about several variants of that mistake.

The project lives at dotnet/BenchmarkDotNet, is MIT licensed, has around 11,500 stars and 1,071 forks, and was last pushed on 2026-09-21.

What the library automates before it prints a number

The README organizes its feature list under four headings, and automation is the biggest one. BenchmarkDotNet generates a separate project per job, builds it, launches it in its own process, runs warmup and measurement iterations, and only then reads results back. You never measure a method in the same process that loaded your test infrastructure, which removes a whole category of interference.

Reliability comes from the statistics rather than from repetition alone. The results table has a Mean column and next to it an Error and a StdDev column, and the framework computes confidence intervals through two companion projects maintained by the same author, perfolizer for measurement data and pragmastat for the statistical engine. A difference of two percent with an error of five percent is not a difference, and the table is formatted so you can see that without doing arithmetic yourself.

The framework also inspects the design. It warns about dead code elimination, about setup work happening inside the measured region, and about comparing benchmarks that do not measure the same thing. These warnings are the part people skip and later wish they had read, because a benchmark that never ran the method it claims to measure will report an implausibly good number with total confidence.

Reading the summary table the tool prints

The README shows the output for the MD5 versus SHA256 example, and the shape of that table is the shape of every BenchmarkDotNet result.

| Method |       Runtime |     N |       Mean |     Error |    StdDev | Ratio |
|------- |-------------- |------ |-----------:|----------:|----------:|------:|
| Sha256 |    .NET 4.7.2 |  1000 |   7.735 us | 0.1913 us | 0.4034 us |  1.00 |
| Sha256 | .NET Core 3.0 |  1000 |   3.989 us | 0.0796 us | 0.0745 us |  0.50 |
| Sha256 | NativeAOT 7.0 |  1000 |   4.091 us | 0.0811 us | 0.1562 us |  0.53 |
| Sha256 |          Mono |  1000 |  13.117 us | 0.2485 us | 0.5019 us |  1.70 |

Five columns carry the meaning. Method is what you wrote, Runtime is which job produced the row, N is the parameter value, Mean is the central estimate with Error giving the confidence interval and StdDev the spread, and Ratio is the comparison against the baseline method, which is why one method is marked `baseline: true`.

The example also shows why parameterization matters when reading a result. At N of 1000 the MD5 method sits at 0.64 of SHA256 on .NET Core 3.0, and at N of 10000 that ratio moves to 0.90. A single-point benchmark would have reported one of those numbers and implied the other, which is exactly the kind of claim that does not survive review.

Results export to Markdown, HTML, CSV, XML and JSON, and `[RPlotExporter]` adds an R plotting script for charts.

Runtimes, languages and architectures it will run on

The support matrix in the README is long and worth reading literally, because it defines what a job attribute can ask for. Supported runtimes are .NET 5 and later, .NET Framework 4.6.1 and later, .NET Core 2.0 and later, Mono, and NativeAOT. Supported languages are C#, F# and Visual Basic, so this is not a C#-only tool despite the sample being C#.

Operating systems covered are Windows, Linux and macOS. Architectures listed are x86, x64, ARM, ARM64, Wasm and LoongArch64, which is a broader set than most people expect and explains why the WASM trimming bug fixed in 0.15.8 mattered.

Each of those combinations is a separate job, and each job means a separate build and process launch, which is the cost of the reliability guarantee. A benchmark that sweeps four runtimes and two parameter values is eight builds. That is why the README frames simplicity as one of its four design aspects, and why trimming the job list is the normal first act of adapting the example to your own code.

Compile-time analyzers arrived in 0.15.7

Version 0.15.7, published 2025-11-12, is the release that changed how you find mistakes. It added Roslyn analyzers that validate BenchmarkDotNet usage at compile time, checking that the benchmark class is public and non-sealed, that generic constraints are declared, that `[Arguments]`, `[Params]` and `[ParamsAllValues]` are used correctly, that `[GenericTypeArguments]` requirements are met, that only one baseline method exists per category, and that `BenchmarkRunner.Run` is invoked properly.

The same release improved .NET Framework version detection by reading the version from `TargetFrameworkAttribute` rather than inferring it, and it bumped Perfolizer to 0.6.1 for updated Windows and macOS detection.

Version 0.15.6 a week earlier added ref struct parameter support for `[ArgumentsSource]`, so a benchmark can take `Span<T>` or `ReadOnlySpan<char>` parameters, fixed Native AOT runtime moniker normalization, and upgraded to Perfolizer 0.6.0 with the Pragmastat statistical engine integration. Version 0.15.8 on 2025-11-30 added an OpenMetrics exporter for Prometheus-compatible metrics output, made the Roslyn analyzers multi-target with better type checking, and fixed a process deadlock when reading process output along with the WASM generated project being trimmed out. That deadlock fix is the one to note if you have had a benchmark hang for no visible reason.

Where the README ends and the documentation site starts

The README is a good advertisement. It makes the adoption claim, shows one complete example, shows the table that example produces, lists the support matrix, and then opens the feature section with four headings. It is not an attribute reference.

Everything else is on benchmarkdotnet.org, linked from the top navigation bar: a getting started guide with a copy-pasteable version of the example, an overview document, and per-feature pages. The parameterization and baseline pages are linked directly from the feature prose, which tells you where the interesting material lives.

Adoption is the other thing worth taking from the README. It claims use in 27,400 or more GitHub projects and names the .NET Runtime, the Roslyn compiler and the dotnet/performance repository as adopters. That matters for a library whose output other people read: when the runtime team changes how it measures something, BenchmarkDotNet tends to follow, because they are the same audience.

Editorial conclusion

BenchmarkDotNet is the reference answer to microbenchmarking in .NET, and its value is less about speed than about honesty: it builds a separate project per runtime, warms the code up, forces enough iterations for the numbers to mean something, and prints an error column next to every mean so a reader can see whether a difference is real. Version 0.15.x added Roslyn analyzers that catch malformed benchmarks at compile time and an OpenMetrics exporter for Prometheus, both of which remove a class of mistake that used to surface only as a confusing table. What the README does not cover is the attribute surface in depth, which lives on benchmarkdotnet.org under features and guides. Start with the MD5 versus SHA256 example from the README, delete the runtime attributes you do not need, and read the getting started guide before trusting a single ratio.

Frequently asked questions

Official sources

1. dotnet/BenchmarkDotNet on GitHub
2. License: MIT
3. Project website
4. README
5. Releases