GCToolkit: a Java library that turns HotSpot GC logs into an event stream you can query

From log lines to an event stream you can aggregate over

GCToolKit is a set of libraries for analyzing HotSpot Java garbage collection log files. The toolkit parses GC log files into discrete events and provides an API for aggregating data from those events, which allows the user to create arbitrary and complex analyses of the state of managed memory in the JVM represented by the garbage collection log.

That two step description, parse then aggregate, is the whole design. A GC log is a stream of text with timestamps, and reading it in a text editor tells you what happened in one run. Turning it into discrete events with typed fields means you can sum pause durations by collection type, count promotions, or compare a baseline run against a candidate, and that is the difference between looking at a log and analysing one.

The framing the README puts around it is worth repeating because it explains why a library is warranted at all. Managed memory in the JVM is composed of three main pieces: memory buffers known as Java heap, allocators which perform the work of getting data into Java heap, and garbage collection. While GC is responsible for recovering memory in Java heap that is no longer in use, the term is often used as a euphemism for memory management, and the phrases tuning GC or tuning the collector are used with the understanding that they refer to tuning the JVM's memory management subsystem. The best source of telemetry data for tuning GC comes from GC logs, and GCToolKit has been helpful in making this task easier by providing parsers, models and an API to build analytics with.

The consequence of the euphemism is that the collector is only one of the three pieces, and a log that shows long pauses does not by itself say whether the problem is the collector choice, the allocation rate, or the heap sizing. An event model lets you separate those, which is the analytical value the library adds over a log viewer.

Three Maven artifacts, and why the split matters for your build

The GCToolKit artifacts are in Maven Central, and the README says you will need to add the api, parser and vertx modules to your project in the dependencyManagement and/or dependencies section as you see fit.

<dependency>
    <groupId>com.microsoft.gctoolkit</groupId>
    <artifactId>api</artifactId>
    <version>3.0.4</version>
</dependency>

<dependency>
    <groupId>com.microsoft.gctoolkit</groupId>
    <artifactId>parser</artifactId>
    <version>3.0.4</version>
</dependency>

That last artifact is the interesting one. The repository layout explains it: api/, parser/ and vertx/ are the three top level source directories, alongside the sample/ project, the gclogs/ directory holding test log files, and the root pom.xml that aggregates them. The vertx artifact is the third module the README tells you to add alongside api and parser.

So the split is functional, not cosmetic. The api module gives you the event model and the aggregation API, the parser module gives you the log readers, and the vertx module is the third artifact the README asks you to add. A project that only needs to parse a small log synchronously may not want the whole stack, and the README's phrase as you see fit is an invitation to pick.

The repository also carries the usual infrastructure of a project run under Microsoft's open source process: a CODE_OF_CONDUCT.md, a CONTRIBUTING.md, a CONTRIBUTORS.md, a SECURITY.md, a SUPPORT.md, a DEVELOPERS_GUIDE.md, a NOTICES.md and a.devops/ directory. The.mvn/ directory and the mvnw and mvnw.cmd wrappers mean the build does not require a system Maven, and.devcontainer/ gives a containerised development environment. There is a pmd-ruleset.xml and a spotbugs-excludes.xml, so static analysis is part of the build, and the IT/ directory is where the integration tests live.

The sample project, and the analysis the README points you at

The documentation keeps its example short, and the example is a specific analysis rather than a hello world.

You can run the Maven project HeapOccupancyAfterCollectionSummary sample as an example of this. The sample project has its own README at sample/README.md, and the repository carries sample/pom.xml and sample/src/.

That sample name tells you what the library is for. Heap occupancy after collection is the number that answers whether your collector is doing its job, because a heap that is still nearly full after a full collection means the live set is close to the heap size and no collector configuration will save you. In a log viewer you can find a full collection and read the number by eye, but if you have a thousand runs, or want the number per collection type, or want to compare it across a change to your heap flags, you need to parse the events and aggregate them.

That is the workflow the library enables and the reason it exists as an API rather than a report generator. The point is not that GCToolKit will tell you what to do about your GC settings. The point is that it will give you numbers you can put in a dashboard, a test assertion, or a before and after comparison, computed the same way every time.

The repository makes the same point in its structure. The gclogs/ directory at the top level holds the log files used as test fixtures, which is a reminder that the parser's hard problem is not parsing but handling the several incompatible log formats HotSpot has used over the years and the configuration flags that change the output. A parser that quietly misreads a format is worse than no parser, so the fixture corpus is the real specification of what it handles.

Why an event model beats a log viewer for aggregate questions

The comparison worth making is not against other Java libraries, because there is not really a competing category here. It is against the two things people actually do today, which are grep and a log viewer.

A log viewer is the better tool for a single run. It renders the timeline, correlates the events into pauses, and lets you scroll. Nothing beats that for understanding one specific pause.

Grep is the better tool for a quick check, and the reason GCToolKit exists is that neither scales to the question a performance engineer actually needs answered. The questions that matter are comparative: is this pause time new, is it promotion or collection, does it happen only on certain heap sizes, is the young generation the problem or the old one. Answering those from raw text means writing regular expressions against a format that changes between JDK versions, and those regular expressions are exactly the thing that breaks silently when a flag changes.

The event model inverts that. Once a pause is a typed object with a duration and a cause, the aggregation is ordinary Java code over a collection, and the fragility moves to the parser, which is where it belongs and where the fixture corpus lives.

The cost of that design is that you write a program rather than run a command. There is no command line here, no report to open, nothing to click. For a one off investigation the fastest path is still the log viewer, and adopting this toolkit means accepting that you are building a small analysis program rather than installing a tool. The repository's own answer to that, the sample directory and the discussion tab, is a starting point rather than a finished product.

What the toolkit cannot tell you

Three limits, and the first is the important one.

The first is that a GC log is a partial view. It records what the collector did, not what your application allocated, and a log line that says a collection took 200 milliseconds says nothing about which of your objects were live. The README is careful about this in the framing, since the three pieces of managed memory are the heap, the allocators and the collector, and a GC log is telemetry about one of the three. If your problem is that your application allocates too much, the toolkit will faithfully report the consequences and stay silent about the cause.

The second is that the model is HotSpot specific. The name of the project says it, the parsers are for HotSpot log formats, and the event types describe HotSpot collection phases. This is not a general JVM analysis toolkit, and there is no OpenJ9 or GraalVM story in the project documentation.

The third is the operational one. A GC log is large, often very large, and it is produced on a machine that was already under stress. Collecting and shipping the logs to where the analysis runs is a problem the library does not solve, and the vertx module exists precisely because reading these files efficiently is hard. If your logs live on a production host and your analysis runs in CI, the pipeline around the toolkit is where most of your engineering time goes.

None of that makes it a poor fit for what it does. It makes it a component rather than a product, and a component needs a reason to be a dependency.

Editorial conclusion

GCToolkit fits a Java team that has to answer questions about GC behaviour across many runs, such as whether a pause is caused by promotion failure or by a fragmented old generation, where a log viewer shows you one file and the toolkit lets you aggregate across a corpus. It does not fit a developer who wants to read a single log quickly, since a log viewer is faster for that, and it does not fit a team outside the JVM, because the model is HotSpot specific. Check three things before you adopt it. Check the log formats your JVM actually produces, because the parser exists to handle the different HotSpot logging formats and your JDK version determines which one you get. Check the module split, because api, parser and vertx are separate artifacts and adding all three pulls in the Vert.x based concurrency layer whether or not you need it. And check the version you pin, because the README's dependency example still shows 3.0.4 while the latest listed release is 3.7.0, so the documented coordinates are behind the code. The licence is MIT, and the project is developed under the Microsoft open source process with a CODE_OF_CONDUCT.md and SECURITY.md in the tree.

Frequently asked questions

Official sources

1. License: MIT
2. microsoft/gctoolkit on GitHub
3. Project website
4. README
5. Releases