DataFixerUpper: Minecraft's data migration engine, usable as a library

Why Minecraft needed a data migration engine at all

DataFixerUpper is described as a set of utilities designed for incremental building, merging and optimization of data transformations, and it was created for converting the game data for Minecraft: Java Edition between different versions of the game.

The problem underneath is a classic one that happens to be very visible in a game. The world data is stored in files, the format of those files changes between releases, and the game has to read a world saved by an older version. The obvious approach is to write a converter that reads the old format and writes the new one, and then to write another one for the version after that, and another, and another. The cost compounds because each converter is written against a specific pair of formats, and the number of pairs grows with the square of the number of versions.

The approach here is different. Instead of a converter per version pair, there is a set of schemas, one per data version, and a set of fixes, each a rewrite rule from one type to another. Because the fixes compose, a world saved at version 1 can be brought to version 5 by applying the chain of fixes between them, and a fix written for a transition in the middle can be reused by every path that passes through it. The result is a library rather than a script, and the data structures are the library's public API.

For a reader outside the game, the interesting thing is that this is a rare, well documented, MIT licensed example of a real production data migration system with an academic pedigree. The intended users are of course Minecraft version implementers, but the design generalises to any system where a stored format has a long life and many versions.

Schema, DataFix and the builder that compiles them

The core data types are Schema and DataFix, and the README defines them precisely.

Schema is a set of type definitions specifying what data types the system is interested in and how they relate to each other. DataFix is a rewrite rule between types. DataFixerBuilder takes a list of schemas and fixes, converting between those schemas, and creates an optimized converter between the types described in those schemas. DSL is a class with building blocks used to create schemas and fixes.

So the three-part model is: a schema describes a shape, a fix describes a change of shape, and the builder takes the lot and produces a single converter that can go from any input schema to any output schema, with the intermediate steps resolved for you.

The word optimized in the builder's description is the one that carries weight, and the References section explains where the technique comes from. The links are to work on point-free program transformation, specifically Cunha and Pinto from 2005, and on the essence of strategic programming by Lämmel, Visser and Visser from 2002. Strategic programming is the idea that a data transformation can be written as a composition of small, generic strategies, and the compiler can then fuse them.

That is the whole trick behind the incremental building in the project description. A naive chain of a hundred rewrite rules applies a hundred transformations to your data, each one walking the whole structure. An optimizing builder can look at the chain and generate code that goes straight from the input shape to the output shape, touching each part of the data once.

The DSL class is what makes the schemas and fixes writable at all. Building type definitions and rewrite rules by hand in Java is verbose, and a DSL is the standard answer, and the fact that the README names it as a first class part of the API rather than an internal detail tells you the authors expect it to be used.

Adding the dependency from the Minecraft libraries repository

It is published on the Minecraft libraries repository, and the README gives the Gradle and Maven forms.

For Gradle, first include the repository:

maven {
    url "https://libraries.minecraft.net"
}

and then use the library, changing the placeholder to the latest version:

compile 'com.mojang:datafixerupper:(the latest version)'

The Maven form is the same idea with a repository block and a dependency:

<repository>
  <id>minecraft-libraries</id>
  <name>Minecraft Libraries</name>
  <url>https://libraries.minecraft.net</url>
</repository>

<dependency>
    <groupId>com.mojang</groupId>
    <artifactId>datafixerupper</artifactId>
    <version>(the latest version)</version>
</dependency>

The coordinates are `com.mojang:datafixerupper`, and the placeholder in the README is a literal instruction to substitute the current version, which is worth doing carefully because the version line tells you how much of the fix corpus you are getting.

Two small things to note. The Gradle snippet uses the old `compile` configuration rather than the modern `implementation`, which tells you the README has not been updated for recent Gradle, and the build in this repository is the older `build.gradle` with `settings.gradle` and the Gradle wrapper committed. And the repository is a Gradle project with `gradlew` and `gradlew.bat` at the root, so building it from source needs a Gradle wrapper invocation rather than a system Gradle.

The licence is MIT, with the LICENSE file at the root and the licence badge in the README header, so embedding the library in a commercial project is straightforward. The project also carries a Microsoft Contributor License Agreement requirement for contributions, with details at cla.microsoft.com, and has adopted the Microsoft Open Source Code of Conduct.

The academic lineage behind the design

The References section is unusually long for a utility library, and it is worth reading as a statement of where the design came from rather than as background reading.

There are four groups. The first is optimizing functions, with the point-free program transformation and strategic programming papers. The second is how to handle recursive types, with a paper on algebraic specialization of generic functions for recursive types, one on generic programming with fixed points for mutually recursive datatypes, and a formal comparison of approaches to datatype-generic programming. The third is optics, with the profunctor optics paper and the paper on generic point-free lenses. The fourth is tying it together, with the type-safe two-level data transformation paper, the transformation of structure-shy programs paper, and the paper on calculating with lenses for optimising bidirectional transformations.

Three themes run through them. The first is the treatment of recursive types, which is the hard part of any schema versioning scheme. A game world contains structures that contain themselves, and a fix that renames a field inside a recursive structure has to terminate and to mean the same thing at every depth. The datatype-generic programming work is what makes the schema system able to express that at all.

The second is the structure-shy transformation idea, which is the conceptual heart of the design. A transformation that is structure-shy does not care about the shape of the data it walks. A fix that renames a field in a chunk is expressed as a rule about that field, and the machinery applies it wherever the field appears, at whatever depth, without the fix author writing a traversal.

The third is bidirectionality. Most of the lens literature is about getting a value out of a structure, and the last reference in that group is about using lenses to optimise bidirectional transformations, which is exactly what a data version converter is: a transformation you may want to run forwards, and sometimes backwards.

The practical consequence for a user is that the library's behaviour in edge cases is defined by published theory rather than by whatever the implementation happened to do, which is a good property for a component that rewrites your data on upgrade.

A 2018 release against a 2026 branch

The version history is the thing to look at first here, and it needs care.

The recent releases are 1.0.20 from 2018-10-16 and 1.0.19 from 2018-10-05, both tagged on the `master` branch, which is also the default branch. The last push to that branch was on 2026-10-09, and the repository is not archived.

So the source tree is being worked on, and the published artifact is years behind the tip. If you resolve `com.mojang:datafixerupper` from the Minecraft libraries repository, you get the version whose fixes stop at whatever the 2018 schema set covered. If you build from the branch, you get a much larger fix corpus and a codebase that has had eight years of changes without a release to match.

That gap has an obvious cause in a game that ships major versions every few months: the fix corpus grows with every data format change, and shipping a new artifact version for every batch of fixes is a decision somebody has to make. Whether that decision has been made recently is not visible from the repository, and the absence of a release since 2018 is a fact rather than a conclusion.

There is also a licensing-adjacent wrinkle for contributors that users never see. Most contributions will require you to agree to a Contributor License Agreement, declaring that you have the right to grant the rights to use your contribution, and the project has adopted the Microsoft Open Source Code of Conduct with a dedicated FAQ and a contact address for questions about contributions. Those are Microsoft conventions from the acquisition, and they are worth knowing about before you open a pull request.

The MIT licence on the code itself is unaffected by any of that.

Where DataFixerUpper is the wrong tool

There are three cases where this library does not fit, and the first is a matter of scope rather than quality.

The first is general purpose data movement. DataFixerUpper is built for a specific shape of problem: a bounded, known set of schema versions with fixes between them, and a requirement that the conversion be exhaustive and correct for every field. If your need is to load a CSV into a warehouse, or to move records between two arbitrary systems, this is a framework with a lot of ceremony for a job that needs none of it.

The second is a schema you do not control and do not know. The whole model assumes you can enumerate the versions and write the rules between them. If your data arrives in whatever shape the producer felt like that day, there is no schema set to write, and you are back to writing code that inspects and repairs.

The third is the maintenance question, and it is the one that actually decides it. A library whose published version is 1.0.20 from 2018 has a fix corpus that ends in 2018. A Minecraft world saved by a recent version needs fixes written after that date, and if the artifact on the repository is the 2018 one, the migration will not be complete, and it will fail in a way that looks like corrupt data rather than a missing rule. That is a hard failure to diagnose unless you already know the schema history.

The alternative approach, for comparison, is a hand written migration per version pair with a chain of converters. The difference in approach is that DataFixerUpper makes you write rules per transition, once, and lets a builder resolve and optimise the path, while the hand written approach writes a full converter per pair and composes them yourself, with the traversal cost at every step. For a handful of versions the hand written approach is quicker to start, and it stays legible, because you can read one converter end to end. The library pays off as the version count grows, which is exactly the situation Minecraft is in and exactly the situation most schema-migrating projects are not yet in.

If you do adopt it, borrow the model rather than the artifact. A Schema, a DataFix and a builder that composes them is a good design for any project with a long-lived stored format, and reimplementing that pattern in your own language with your own fix corpus is a smaller commitment than depending on a library whose release history has stopped.

Editorial conclusion

Adopt DataFixerUpper if you maintain a dataset whose schema changes over releases and you want the migration to be expressed as typed rewrite rules rather than ad hoc code, since the Schema, DataFix and DataFixerBuilder model is the part worth borrowing. Do not adopt it as a general purpose ETL engine, and do not expect the published artifact to be recent, because the newest release is 1.0.20 from 2018-10-16 even though the repository was pushed to on 2026-10-09. Verify three things first: which version of the artifact you actually resolve, because the published tag and the current branch may differ by years of work, that the fix graph in the artifact covers the schema changes you need to make, and that the DataFixerBuilder output is deterministic for your data, since a converter that merges two paths can behave differently from a hand written migration.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Mojang/DataFixerUpper on GitHub
4. README
5. Releases