microservices-framework-benchmark: the raw numbers behind the light-4j claims

The measurement is a Hello World handler, not a service

The project description is precise about what it measures: raw benchmarks on throughput, latency and transfer of Hello World on popular microservices frameworks. That word Hello World is the most important thing in the whole repository, and everything else follows from it.

A Hello World handler returns a fixed string. It does not read a database, serialise a domain object, call a downstream service, authenticate a request or write to a log. What the benchmark therefore measures is the framework's own request handling path: routing, the filter or middleware chain, serialisation of the response, and the HTTP server underneath. It is a measurement of framework overhead, and it is a legitimate thing to measure, but it is not a measurement of what a real endpoint costs.

The harness is wrk with a Lua script. The script is pipeline.lua, located at microservices-framework-benchmark/pipeline.lua, and it generates more requests per second. The wrk invocation is consistent across the runs, four threads and 128 connections for 30 seconds against localhost:8080, with latency statistics requested and the pipeline script attached.

wrk -t4 -c128 -d30s http://localhost:8080 -s pipeline.lua --latency -- / 16

The trailing path argument and number vary, and the README explains why: the point of the pipeline script is to push more requests per second, and the light-4j section shows the same command with a different value producing much higher throughput but also server errors in some other frameworks. The number after the path is the number of pipelined requests per connection.

The primary language of the repository is recorded as C++, which is a bit surprising until you look at the top level listing and find the node-uws directory, a Node.js server built on the uWebSockets native library.

Forty framework directories, each a runnable server

The value of this repository is that it is not a spreadsheet of numbers. The top level is a directory per framework, and each one contains the source of a server you can start yourself.

The Java entries dominate the list: light-4j, actframework, spring-boot-3-tomcat, spring-boot-3-undertow, spring-boot-reactor, spring-boot-tomcat, spring-boot-undertow, spring-boot2-tomcat, spring-boot2-undertow, spring-boot2-webflux-netty, spring-boot2-webflux-undertow, spring-vertx, vertx, jooby, jooby-2.x, dropwizard, spark, ratpack, jfinal, play-java, play1, micronaut-reactor, micronaut-rxjava, msf4j, helidon, quarkus, undertow, baseio, ninjaframework, pippo-jetty, pippo-tomcat, pippo-undertow, seedstack-filter, seedstack-jersey2, bootique-jetty, akka-http, wildfly-swarm, payara-micro and rapidoid.

The non-Java entries are the reason the list is interesting rather than a Spring shootout. Go has go-fasthttp, go-http, go-httprouter and iris. Node has node-express, node-loopback and node-uws. There is a dotnet directory, and directories for javalin and jfinal-style alternatives.

Each example is a small server you start with a couple of commands. The Node Express example is the simplest.

cd node-express
npm install
node server.js

And the Go standard HTTP one is a single preforked process.

cd go-http
go run server.go -prefork

So the repository is simultaneously the benchmark and its own reproduction environment. If a figure in the table looks wrong to you, you can start that framework yourself and re-run the same wrk command against it, which is the single most valuable thing a raw benchmark repository can offer.

The documentation is hosted on GitHub Pages at the networknt documentation site, there is a CHANGELOG.md and a CONTRIBUTING.md at the root, and the project links to the Stack Overflow tag, a Google group, a Gitter chat, a subreddit and a YouTube channel for the light-4j ecosystem, which tells you who the surrounding community is.

The ranking table is a single snapshot with one line of context

The table in the README is what people cite, and it needs the same caution as any self-published benchmark.

It has five columns: framework, language, max throughput, average latency and transfer. The top of the table has Go FastHttp with a max throughput of 1,396,685.83, Light-4j in Java at 1,344,512.65, ActFramework at 945,429.13, Go Iris at 828,035.66, Vertx at 803,311.31, then Node-uws at 589,924.44 and Dotnet at 486,216.93. The tail is Dropwizard at 33,819.90, Spring Boot Tomcat at 33,086.22, Node-Loopback at 32,091.95, Payra Micro at 24,768.69 and WildFly Swarm at 21,541.07.

The top-level numbers are worth pausing on. Go FastHttp and Light-4j are effectively tied on throughput, and Light-4j has an average latency of 2.36ms against Go FastHttp's 99.98ms. The Java frameworks spread from ActFramework at 2.22ms down to the Spring Boot entries at 38.94ms and 82.93ms.

The column headers do not say when the table was generated, on what hardware, with which JDK, or whether every entry was measured under the same conditions. The repository is MIT licensed and the last push was on 2026-09-18, so the table and the code are in the same state, but the table itself carries no date stamp.

There is one more thing to weigh, and it is not a criticism of the measurements. The project lives under the networknt organisation, and light-4j, which sits second in the table, is networknt's own framework. The comparison includes the author's own project alongside competitors, and the README is explicit that it is showing what the author's framework does. That is not the same as a neutral third-party comparison, and the honest way to read the table is as a well-instrumented set of raw numbers produced by someone with a stake in one of the entries.

The detailed sections that follow the table are more informative than the summary precisely because they include the full wrk output rather than a number extracted from it.

The raw output shows why the averages mislead

The detailed sections publish complete wrk runs, and reading one closely is instructive.

The light-java run at the higher request rate is the headline result.

Running 30s test @ http://localhost:8080
  4 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   774.30us    1.77ms  40.31ms   91.51%
    Req/Sec     0.93M    81.46k    1.06M    87.83%
  Latency Distribution
     50%  286.00us
     75%  529.00us
     90%    1.92ms
     99%    0.00us
  110603088 requests in 30.01s, 12.88GB read
Requests/sec: 3685234.33
Transfer/sec:    439.31MB

Then the same server pushed harder, where the README notes that some other frameworks get server errors.

Running 30s test @ http://localhost:8080
  4 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.35ms    2.80ms  54.23ms   89.24%
    Req/Sec   550.10k    64.34k    1.22M    74.54%
  Latency Distribution
     50%    1.56ms
     75%    2.87ms
     90%    5.41ms
     99%   22.17ms
  65803650 requests in 30.10s, 6.50GB read
Requests/sec: 2186203.44
Transfer/sec:    221.00MB

And the Spring Boot comparison runs on the same command line.

Running 30s test @ http://localhost:8080
  4 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    82.93ms  108.77ms   1.58s    89.45%
    Req/Sec     8.40k     3.68k   22.19k    68.54%
  Latency Distribution
     50%   45.66ms
     75%  101.59ms
     90%  197.72ms
     99%  542.87ms
  995431 requests in 30.09s, 119.79MB read
Requests/sec:  33086.22
Transfer/sec:      3.98MB

Three things stand out. The average latency of 774 microseconds in the first run is an order of magnitude better than the 2.35 milliseconds in the second, on the same server, because the server is being asked to do far more. The throughput figures of 3.68 million and 2.19 million requests per second are for the same framework and differ by a factor of nearly two across the two runs, which tells you how much the pipelining depth changes the result. And the 99th percentile figure of 0.00us in the first run is a measurement artefact, since a percentile that rounds to zero means the tail is not being sampled meaningfully at that depth, so the percentile columns should not be read as precise.

The Node Express section publishes the same shape of output, and the Go standard HTTP section begins but is truncated in the visible documentation. The project also states that Spring Boot with embedded Undertow was added for a closer comparison, since light-4j uses Undertow core only, and reports the ratio between the two as about 33 times rather than 44.

Where this benchmark is the wrong instrument

The first wrong-tool case is choosing a framework from the table. A Hello World handler is the floor of a framework's cost, not its typical cost. Once your application serialises objects, talks to a database, calls a downstream service or runs a filter chain with authentication, the relative ordering can change completely, and the framework you picked for its Hello World number may be irrelevant to your actual throughput.

The second case is a deployment that is not a single local process on loopback. These are localhost measurements with 128 connections and 4 client threads against a server on the same machine. A real deployment has network latency between client and server, which changes what the latency column means and can move the ranking toward whichever framework handles partial responses and connection reuse better.

The third case is a memory-constrained environment. The table has no memory column at all, which is a real gap. Two frameworks can produce the same requests per second while differing substantially in resident memory, and a container with a hard limit will make that difference the binding constraint. The throughput figure in isolation is not a selection criterion.

The fourth case is a need for a neutral comparison. This benchmark is published by the organisation that maintains one of the frameworks it measures, and the documentation is enthusiastic about that framework. The raw data is genuinely useful, and you can re-run it, but a table whose top entry is your own project deserves a second opinion before it drives a decision.

On licensing, the project is MIT with a LICENSE file at the root, so the code, the per-framework examples and the pipeline script are all reusable in your own harness. The last push was on 2026-09-18 and there are no GitHub releases, so you follow the master branch rather than a tag. The CHANGELOG.md at the root is where changes to the harness are recorded.

Editorial conclusion

This benchmark is useful to a reader who wants to see the raw wrk output rather than a summary chart, and the per-framework source directories make it easy to run one case yourself. It is not usable as a neutral comparison, because the project is published by the networknt organisation, which also maintains light-4j, and light-4j appears near the top of its own table. Before citing any figure, check which framework and which server combination produced it, note that the test drives a Hello World handler that does no application work, and re-run the case on your own hardware rather than treating the published throughput as a property of your stack.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. networknt/microservices-framework-benchmark on GitHub
4. README