lambda-perf: a daily Lambda cold start benchmark you can read

The complaint is that cold start articles go stale

The stated motivation is narrow and specific. There are already a lot of blog posts about Lambda cold start performance per runtime, and the author could not find any always up-to-date information. So the project exists to fix that by making the data current: the benchmark runs daily, so the numbers reflect the runtimes as they exist now.

That framing explains the shape of everything else. This is not a library you import, and it is not a tool you point at your own function. It is a continuous measurement of the platform itself, published as data.

The primary language is Rust, and the repository root shows why that is a good fit. It is a Cargo workspace with five members, function-trigger-deployer-rs, function-deployer-rs, function-invoker-rs, function-report-log-processor-rs and common_lib, plus a cdk/ directory, an s3-uploader, a container-uploader and a result-builder. The release profile is tuned for size and determinism.

[profile.release]
opt-level = 'z'
lto = true
codegen-units = 1
panic = 'abort'

So the benchmark itself is a set of Rust binaries that deploy functions, invoke them, parse reports and build results, orchestrated by AWS CDK. The licence is Apache-2.0 and the published site is at https://maxday.github.io/lambda-perf.

The runtime matrix is broader than any blog post would test

Step one is a hello-world function written in each AWS supported runtime, and the current list is long. The managed runtimes covered are nodejs20.x, nodejs22.x and nodejs24.x, python3.9 through python3.14, dotnet8 and dotnet10, java11 through java25, and ruby3.2 through ruby3.4.

What is more interesting is the custom runtime list, because that is where the interesting comparison lives. The project builds and packages its own binaries for go, rust and c++ on both provided.al2 and provided.al2023, for dotnet ahead-of-time compilation at versions 7, 8, 9 and 10, for quarkus native and graalvm java17 and java21 and java23, for apple swift 5.8 and 6.2, for bun with and without a layer, for llrt, and for shell on both provided images.

That spread means the benchmark covers the two questions teams actually ask. Which managed runtime starts fastest, and whether compiling ahead of time or shipping a native binary or a shell script beats the managed options. Each function is packaged in a zip file and uploaded to an S3 bucket.

One exclusion is stated plainly. The SnapStart feature is no longer benchmarked, because using it with a simple hello-world application may not represent a meaningful use case. That is the project being careful about what its data means, and it is worth reading as a warning about the whole dataset.

Ten cold invocations a day, with init duration from the REPORT line

Step two is the measurement loop. Every day, each function is freshly grabbed from S3, deployed, and invoked ten times as cold starts. Freshly grabbed matters, because reusing a deployment would measure a warm environment rather than a cold one.

The data captured comes from the REPORT log line, which contains the init duration, the max memory used and other useful information. That report line is saved to a DynamoDB table.

So the measurement is not a wall clock around an HTTP call from outside. It reads the init duration the runtime itself reports, which separates the platform's start-up cost from network and invocation overhead, and it pairs that with the memory the function actually used, which is the number that determines the price tier.

Step three aggregates everything in DynamoDB into a new JSON file and commits it to the repository, with the README pointing at an example file under data/2022/ for a specific day. The data directory in the tree is therefore a dated archive rather than a rolling dashboard, which means you can look at what a runtime did on a particular day rather than only at the current state.

The whole thing is scheduled by the trigger deployer, the same Rust workspace, with the cdk/ directory holding the infrastructure.

The website is a static page over a JSON file

Step four is the presentation layer, and it is deliberately the least interesting part. A static website, hosted on GitHub Pages, fetches the JSON file and displays the result in what the README calls a nice UI. The screenshot in the README is the way most people meet this project, and the repository description says the same thing: Lambda Cold Starts analysis, updated daily.

There is no server, no query layer and no database behind the site. The site reads the committed JSON, and the JSON is the record. That is a useful property for a benchmark: the data behind any number you see on the site is inspectable in git history, so a claim can be checked against the file it came from rather than taken on trust.

Step five is the extension point, and it is the one to pay attention to if you want to contribute. The README asks readers to hack, fork, send a pull request and create their own benchmarks. Combined with the fact that each runtime is a separate packaged function, that means adding a runtime is a matter of adding a function and its packaging, not rebuilding the harness.

The disclaimer at the end of the README is worth quoting in spirit rather than reproducing: the project states that it is not associated, affiliated, endorsed or sponsored by any company, and that no company has reviewed, tested or certified it. For a benchmark measuring a proprietary platform, that is an honest and material caveat.

What this data can and cannot tell you

The limitations are structural, and the README is upfront about the biggest one by dropping SnapStart from the benchmark.

A hello-world function measures the fixed cost of a runtime starting up on a given memory size and architecture. It does not measure your function. If your handler loads a large dependency tree, parses a config file, or connects to a database at import time, your init duration is the platform cost plus all of that, and the second term is invisible here. The benchmark says nothing about steady-state throughput either, since it is measuring start-up, not work.

The second limitation is that the runtime set moves. New runtime versions ship regularly, and the list above already shows four Node versions, six Python versions and four Java versions, which means comparisons have to be made within a version. Looking at a number without knowing which runtime version produced it is a good way to draw a wrong conclusion.

The third is that the platform changes underneath the measurement. The data is only as good as the day it was taken, and a single day of ten cold invocations is a small sample. The value is in the trend and in the freshness, not in any single figure.

What you can take from it is a comparison and a rate of change. Which runtime family starts fastest among the ones AWS supports, whether ahead-of-time compilation or a native binary changes the picture, and whether the gap between options is narrowing as the platform improves.

How it compares to running your own benchmark

The obvious alternative is to benchmark your own function, which is what most teams eventually do. The difference in approach is instructive.

A self-run benchmark measures your code, your dependencies and your memory settings on your account, which is exactly what you need to make a decision about your workload. Its weaknesses are that it takes engineering time, it drifts out of date as dependencies change, and it does not give you a comparable number to check against someone else's claim.

This project inverts all three. It gives you a number you did not have to produce, it refreshes every day, and it is directly comparable across runtimes because every function is the same hello-world. What it gives up is relevance to your specific code, which is why the right way to use it is as a screen: narrow the runtime choice to two or three candidates, then measure your actual handler on those candidates.

There is also a difference in kind from the blog posts it exists to replace. A blog post is a snapshot with a date and an author who may never update it. A daily commit is a series of snapshots, so you can see the direction of travel and notice when AWS changes something about a runtime's start-up behaviour.

The repository is not archived and the last push was on 2026-10-19, with no GitHub releases, since the published artefact is the data and the site rather than a package. The Apache-2.0 licence lets you reuse the data and the harness, and the fork-and-send-a-pull-request instruction means a runtime nobody else benchmarks can be added by anyone.

Editorial conclusion

Adopt lambda-perf if you are choosing a Lambda runtime and want a freshness signal rather than a blog post from last year, and skip it if you need to know how your specific application starts, because the benchmark runs a trivial hello-world function and the README itself drops SnapStart for that reason. Before you quote a number, check the data/ JSON for the date you care about and remember the last push was on 2026-10-19, since the set of benchmarked runtimes changes as AWS ships new ones.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. maxday/lambda-perf on GitHub
4. Project website
5. README