dotnet/performance: the shared benchmark harness for .NET runtimes

Why the benchmarks sit outside the runtime repositories

The README addresses the odd placement head on, and the reason is the interesting part of the project. Finding these benchmarks in a separate repository might be surprising, but performance in a given scenario may be affected by changes in seemingly unrelated components, and a central repository ensures that measurements are made in comparable ways across all the .NET runtimes and repos. That consistency is what lets engineers make progress, and the README adds that it protects customer scenarios.

The runtime coverage is explicit: .NET Core, Full .NET Framework, Mono and NativeAOT. That span matters, because a benchmark that only runs on one runtime tells you about that runtime. Running the same scenario across four means a change can be judged on the axis that matters, whether it is a JIT improvement that helps Core and NativeAOT but not the full framework, or a regression that only appears on Mono.

The primary language of the repository is F#, which fits a benchmark harness: the scenarios are described in a functional style that keeps setup overhead out of the measured region, and the code reads closer to a specification of the workload than to application logic.

How the build table maps branches to pipelines

The README leads with a table of build source versions and their status, which doubles as a map of how the project is validated. Three source versions are listed: main, release/7.0 and release/6.0. Each row carries a public build status and an internal build status, and the link definitions at the bottom of the README show where those point.

The public links resolve to a dnceng-public Azure DevOps project, and the internal links to a dnceng internal project, so the same repository is built by two infrastructures. That split is worth understanding before you interpret a red status: a failure visible on the public badge is the same source version that the internal pipeline also builds, and the release branches are built in parallel, which means a fix has to land on main and then be picked up into a release branch rather than patched in one place.

The repository files match that layout. There is an azure-pipelines.yml at the root for the general build, a separate gc-azure-pipelines.yml, a helix.yml for the machine-based test infrastructure, an .azuredevops/ directory, and an eng/ directory that holds the build engineering scripts. The reports/ directory is where results land, and global.json plus global.net8.json, global.net9.json and global.net10.json pin the SDK versions per target framework, with a corresponding global.net10.json for the newest line.

The repository layout and where the scenarios live

Top level entries show a benchmark project rather than an application. There is a src/ directory for the scenario sources, a scripts/ directory for the helper scripts, an eng/ directory for build engineering, a docs/ directory for documentation, and reports/ for output.

Configuration is spread across the root in a way that reflects how a multi-target harness has to work. Directory.Build.props and Directory.Build.targets apply shared MSBuild properties and targets to every project, NuGet.config points the restore at the package sources the scenarios need, and .config/ holds the dotnet tooling manifest. A pyrightconfig.json sits next to the TypeScript-oriented tooling, and requirements.txt pins the Python side of the build and reporting tooling.

The Python requirements are a decent window into how results move: the Azure Storage blob, queue and identity packages for publishing artifacts, the OpenTelemetry API and SDK for tracing, GitPython and urllib3 for repository access, and cryptography and six as supporting libraries.

cryptography==48.0.1
azure.storage.blob==12.13.0
azure.storage.queue==12.4.0
azure.identity==1.16.1
gitpython<=3.1.41
urllib3==2.7.0
opentelemetry-api==1.23.0
opentelemetry-sdk==1.23.0
six==1.17.0

Those pinned versions are what a local contributor needs installed before the reporting scripts will run.

No install recipe: what the README actually gives you

This is not a project with a get-started script, and the README does not pretend otherwise. There is no clone-and-run sequence, no install command and no sample invocation anywhere in it. The documentation pointer is a signpost rather than a tutorial: the README says to see the documentation signpost at docs/README.md.

That is a fair arrangement for a harness owned by the runtime teams, where the scenarios are run by CI on dedicated machines rather than on a contributor's laptop. The practical route for an outside reader is therefore to read the scenario sources in src/, read the build definitions in eng/ and azure-pipelines.yml, and use the reports/ directory to see the shape of the output rather than to reproduce a run from scratch.

The repository also carries the standard governance files: CODE-OF_CONDUCT.md, a .markdownlint.jsonc and .markdownlintignore for documentation linting, a .gitattributes file, and a .github/ directory. The project has adopted the Contributor Covenant code of conduct, with a link to the .NET Foundation version of it.

Everything here is MIT licensed, with the licence text in a file named LICENSE.TXT, which is the conventional location for it in this organisation.

Where dotnet/performance is the wrong tool

The first limitation is that these are not your workloads. The scenarios are fixed and chosen to exercise runtime internals, so they say nothing about whether your application's p99 latency improved. For a service under load, the relevant measurement comes from your own production traces, not from a benchmark the runtime team wrote to stress a JIT path.

The second is the cost of running them. A serious run wants dedicated hardware, because microbenchmark results move when the machine is noisy, and the presence of helix.yml indicates the results come from a managed fleet of test machines. Reproducing a result on a laptop is possible but not comparable to the published numbers.

The third is scope. The suite measures the runtime, not the ecosystem around it, so a claim that the .NET platform got faster has to be made carefully, because a benchmark regression in one scenario and an improvement in another are both normal and neither generalises.

The fourth is maintenance cost for a consumer. If you want to track the harness over time, the branch table tells you that only main, release/7.0 and release/6.0 are tracked with build status, so a version line outside that set has no visible validation.

Maintenance cadence, licence and who should use the results

The repository is not archived and its last push was on 2026-10-20, so the benchmark set is being maintained with the runtimes it measures. There are no GitHub releases, which fits a repository whose real artefacts are the CI reports rather than downloadable packages. The three tracked source versions in the build table, main and the two release branches, tell you which lines receive active validation.

The licence is MIT, which is permissive enough that an outside team can run the scenarios and publish the results without special permission, provided the notice travels with the code. The .NET Foundation code of conduct sets expectations for participation, and the SECURITY.md file at the root gives the route for reporting security issues.

So the users are the .NET runtime engineers who need a comparable measurement, the contributors who add or tune a scenario, and the engineers outside the team who want to know whether a runtime change helped or hurt. The last group gets the most value from the reports and the least from trying to run the harness themselves, and it is worth being clear about which of those two activities a given question actually needs.

Editorial conclusion

dotnet/performance is the right reference for anyone asking whether a change made a .NET workload faster, because it is the harness the runtime teams use to make that judgement themselves. It is not a benchmarking tool you point at your own application, since the scenarios are fixed and aimed at runtime internals. Verify first that the branch you care about is in the build table, since only main, release/7.0 and release/6.0 are listed with public build status.

Frequently asked questions

Official sources

1. dotnet/performance on GitHub
2. Issues
3. License: MIT
4. README