# google/cluster-data, three generations of Borg workload traces and a format spec > The google/cluster-data repository publishes traces from Google's Borg cluster management software, most usefully the version 3 dataset covering eight cells over May 2019, along with a protobuf schema for the trace format, analysis notebooks and a bibliography of papers that used the data. It is a dataset rather than a tool, and choosing which version to use is the first real decision. **google/cluster-data** — Borg cluster traces from Google - Repository: https://github.com/google/cluster-data - Stars: 1,113 · Forks: 218 - Language: TeX - License: not declared - Published: 2026-10-19 · Updated: 2026-11-03 · Language: en - Canonical page: https://hysenlabs.com/projects/google-cluster-data ## Three trace generations, and the one to use The repository describes various traces from parts of the Google cluster management software and systems, and the Borg workload traces are the main body of it. These are traces of workloads running on Google compute cells managed by the cluster management software internally known as Borg. There are three versions, and the differences between them are not cosmetic. Version 3, documented in ClusterData2019.md and also called ClusterData2019, provides data from eight Borg cells over the month of May 2019. Version 2, documented in ClusterData2011_2.md and also called ClusterData2011, provides data from a single 12.5k-machine Borg cell from May 2011. Version 1, in TraceVersion1.md, is an older and short trace describing a seven hour period from one cell in 2009. The README is blunt about version 1. It is deprecated, and the recommendation is to use the version 2 or version 3 traces instead. Seven hours from one cell in 2009 is a sample of a different machine population and a different software generation, and a scheduling policy evaluated on it is being evaluated on a fossil. So the practical choice is between version 2 and version 3. Version 3 gives you eight cells and a full month, which means you can look at seasonality, at the difference between cells, and at behaviour across a population of clusters rather than one. Version 2 gives you one cell with 12.5k machines, and it predates version 3 by eight years. If your research question is about how a single large cluster behaved, version 2 is still a legitimate dataset, and it has the advantage of being closer to the cluster scales some people care about. The repository also hosts a set of execution traces from ETA, the Exploratory Testing Architecture, which is a testing framework that explores interactions between distributed, concurrently-executing components with an eye towards improving the testing of them. That is a different kind of trace from the workload data and it is not a Borg artifact. ## Power traces and the protobuf schema that makes the data readable Alongside the workload traces the site hosts power traces for 57 power domains during the month of May 2019, documented in PowerData2019.md, and the README states that this trace is synergistic with the ClusterData2019 dataset. Same month, same infrastructure, different physical dimension. That pairing is the interesting part of the repository: it lets you relate what a cluster was doing to how much power it drew, which is a question the workload traces alone cannot answer. The format is documented rather than left implicit. The repository contains clusterdata_trace_format_v3.proto, a protobuf schema for the version 3 trace format, which means the fields of a trace record are specified in a machine readable way rather than only in prose. There is also power_trace_documentation.pdf, a PDF for the power trace documentation. For anyone writing a parser, the schema is the difference between a weekend and a month. A protobuf definition tells you the field numbers, the types and the nesting, and it gives you a canonical implementation to generate code from, so two research groups writing parsers end up with the same interpretation of the same bytes. It is the single most reusable artefact in the repository, more so than the data itself, and it is the one piece you can point at in a paper's methods section. The analysis side is handled with two Jupyter notebooks, clusterdata_analysis_colab.ipynb and power_trace_analysis_colab.ipynb, both with the colab suffix meaning they are meant to be run in Google Colab rather than as local scripts. A notebook in a repository like this is worth reading even if you never run it, because it shows the intended shape of an analysis. ## The bibliography is the citation trail The repository provides a trace bibliography of papers that have used and analysed the traces. The README encourages anyone who publishes a paper using these traces to add it to the bibliography, using a GitHub pull request, which it marks as preferred, or by emailing the bibtex entry to the discussion group address. In either case, the instruction is to mimic the existing format exactly. This is a small thing that matters more than it looks. A dataset like this accumulates a substantial literature, and a bibliography of that literature is how a newcomer discovers what has already been done, which questions are answered, and which are still open. For a dataset of eight cells and a month, the combinatorial space of analysis questions is large enough that somebody has almost certainly asked yours. The file is bibliography.bib, a BibTeX file, which means it can be cited from LaTeX directly, and the primary language of the repository is recorded as TeX, which fits a repository whose published output is papers. There is no repository-wide README section listing what has been found in the data, so the bibliography does that job. The discussion group is the other half of the community infrastructure. It is a low volume group that the README asks you to join so announcements can reach you, and it is where insights, surprises and questions are shared. The group has a few hundred members. The README is unusually specific about how to join: to avoid spammers you must fill out the reason field or the application is rejected, and once a member you can email the group address to announce tools and techniques for decoding the trace data, to share insights, or to ask questions. The asking advice is equally specific: include concrete examples of the issues you run into, screen shots, error codes, and a list of what you have already tried, and do not just say you cannot download the data. ## Licensing that attaches to the data, not just the code The data and trace documentation are made available under the Creative Commons CC-BY licence, and the README states that by downloading the data or using them, you agree to the terms of that licence. The repository carries a Creative Commons CC-BY badge to make that visible. The practical point is that the licence attaches to the data, not only to the text in the repository. If you use the traces in a published paper, the CC-BY terms require attribution, and the natural form of that attribution points at the dataset and at the paper describing it. The repository's own licence metadata is not classified, which is a small inconsistency worth noting if you are auditing dependencies or scanning for licence compatibility in an automated pipeline. The authoritative statement is the README section and the Creative Commons badge rather than the metadata field. The other thing to be aware of is the age of the data. May 2019 and May 2011 are fixed historical snapshots of Google's infrastructure, and there is no release cadence here, no tagged versions and no GitHub releases. The repository is not archived and the last push was on 2026-08-07, so the project still receives changes, but the traces themselves do not get refreshed. If your question is about current cluster behaviour, a 2019 trace answers it about 2019, and the hardware, the workload mix and the scheduler have all moved on since. ## What you have to bring yourself It is worth being clear about what this repository is not. There is no pipeline, no library, no service and no API. There is a set of files, a schema, two notebooks and a bibliography, and everything else is up to you. In practice that means the work starts with downloading the trace data, and then parsing it according to the protobuf schema. A schema and a notebook are a good starting point, but a research project that depends on the trace will need its own parsing and caching layer. The comparison to make is with a synthetic scheduler simulation, which is the standard alternative in this research area. The difference in approach is the entire point of a trace dataset. A simulation gives you a workload you designed, with a distribution you chose, and it tells you how a policy behaves against the workload you imagined. A trace gives you a workload that actually happened, including the irregularities that nobody would put in a simulation, and it tells you how a policy behaves against reality. The tradeoff is that a trace answers questions about one particular month of one particular infrastructure, and a simulation answers questions about the workload model you built. The pragmatic path for a newcomer is to start with the version 3 traces, read ClusterData2019.md for the field semantics, use clusterdata_analysis_colab.ipynb to see the intended analysis, and check the bibliography before deciding what to look for. If the question involves power rather than scheduling, the 57 power domains from the same month are the corresponding dataset. ## Conclusion google/cluster-data suits researchers who need realistic large-scale cluster scheduling data, since the version 3 traces cover eight Borg cells over a month and carry a documented protobuf schema plus worked notebooks, and it suits people who want to cite prior work, since the bibliography tracks papers that analysed the same traces. It is the wrong choice if you want live production telemetry, since the data is a fixed historical snapshot with no update cadence, and it is the wrong choice if you need a ready-made parser, since the repository ships the schema and analysis notebooks but not a maintained client library. Before starting, read ClusterData2019.md rather than the older TraceVersion1.md, which the README marks deprecated, and check the CC-BY terms, since downloading or using the data constitutes agreeing to them. ## FAQ ### What is in the google/cluster-data repository? It holds traces from parts of the Google cluster management software and systems, mainly Borg workload traces, plus a set of ETA execution traces and power traces for 57 power domains from May 2019. The Borg versions are version 3 (eight cells over May 2019), version 2 (a single 12.5k-machine cell from May 2011) and a deprecated version 1 from 2009. ### Which version of the Borg traces should I use? The README marks version 1 as deprecated and strongly recommends version 2 or version 3. Version 3 gives eight cells over a month, version 2 gives a single 12.5k-machine cell from May 2011, and the documentation for each is in ClusterData2019.md and ClusterData2011_2.md. ### How do I parse the trace data? The repository ships clusterdata_trace_format_v3.proto, a protobuf schema for the version 3 trace format, and two analysis notebooks, clusterdata_analysis_colab.ipynb and power_trace_analysis_colab.ipynb, which are meant to be run in Google Colab. The repository contains no parser code, only the schema and the notebooks. ### Can I cite papers that used these traces? Yes. The repository provides a trace bibliography in bibliography.bib of papers that have used or analysed the traces, and the README asks anyone who publishes such a paper to add it through a pull request or by emailing the bibtex entry, mimicking the existing format exactly. ### What licence applies to the data? The data and trace documentation are made available under the Creative Commons CC-BY licence, and the README states that downloading or using the data means agreeing to those terms. ## Sources - [google/cluster-data on GitHub](https://github.com/google/cluster-data) - [Issues](https://github.com/google/cluster-data/issues) - [README](https://github.com/google/cluster-data/blob/master/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/google-cluster-data