# Models Genesis: self-supervised pre-training for 3D medical images

> Models Genesis learns representations from unlabeled CT and MRI volumes by making the model reconstruct deliberately distorted images, and it ships the pre-trained weights in both Keras and PyTorch form. It is the code behind two award-winning papers, and the practical catch is a dependency set pinned to TensorFlow 1.14 and PyTorch 1.3.1.

**MrGiovanni/ModelsGenesis** — [MICCAI 2019 Young Scientist Award] [MedIA Best Paper Award] Models Genesis: self-supervised pre-training for 3D medical images. Learns transferable representations from unlabeled CT and MRI volumes, then fine-tunes for downstream segmentation and classification. Keras and PyTorch weights included.

- Repository: https://github.com/MrGiovanni/ModelsGenesis
- Website: https://www.cs.jhu.edu/~zongwei/publication/zhou2020models.pdf
- Stars: 796 · Forks: 143
- Language: Jupyter Notebook
- License: NOASSERTION
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/mrgiovanni-modelsgenesis

## What the framework does: learn anatomy from broken images

Models Genesis is a self-supervised learning framework that pre-trains 3D models on unlabeled medical images. It learns transferable visual representations from CT and MRI volumes with no manual annotation, and those representations fine-tune for downstream segmentation and classification.

The mechanism is image restoration as a proxy task. Pre-training uses non-linear intensity transformation, local pixel shuffling, out-painting and in-painting, and the model learns anatomy by reconstructing deliberately distorted volumes. In the self-supervised learning literature this places Models Genesis in the generative or restorative family, as distinct from contrastive methods, and that is the single most useful thing to know about it if you are choosing between approaches.

The comparison with contrastive methods is a real difference in what the model is asked to do. A contrastive method is trained to tell similar and dissimilar patches apart, while a restorative one is trained to put a damaged volume back together. The restored output has to carry enough information about the original anatomy that the encoder is forced to represent it.

The original name of the project, Generic Autodidactic Models, describes the same idea in three parts. Models are created ex nihilo, with no manual labeling. They are self-taught, learned by self-supervision. And they are generic, serving as source models for application-specific target models.

The repository describes this line of work as an early example of what would later be called a foundation model for medical imaging, and its topic list uses the foundation-models tag, which is a good way to place it among current approaches without having to accept the label as marketing.

## Two papers, two venues, one codebase

The repository is the official implementation for both the training and the use of the pre-trained model, and it covers two separate publications.

The first is Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis, presented at the International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI, in 2019, with a Young Scientist Award. The author list is Zongwei Zhou, Vatsal Sodha, Md Mahfuzur Rahman Siddiquee, Ruibin Feng, Nima Tajbakhsh, Michael B. Gotway and Jianming Liang, with the affiliation Arizona State University and Mayo Clinic.

The second is the journal version, Models Genesis, in Medical Image Analysis, volume 67, pages 101840, in 2021, with Jiaxuan Pang added to the author list, and it received the MedIA Best Paper Award. The DOI is 10.1016/j.media.2020.101840.

The same repository therefore covers the conference paper and the extended journal paper, which is unusual and convenient: a method published twice with more experiments still ships as one codebase.

A third piece of related work is also cited here, a PhD thesis by Zongwei Zhou from Arizona State University in 2021 titled Towards Annotation-Efficient Deep Learning for Computer-Aided Diagnosis. The citation section gives all three entries in BibTeX, and there is a CITATION.cff file at the repository root for citation tooling. If you use the code or the pre-trained weights for research, the README asks you to cite the papers.

## What the repository actually contains, directory by directory

The top level is small enough to describe in full, which makes it easy to see what kind of artifact this is.

There are two parallel implementations. The keras/ directory and the pytorch/ directory hold the model code and the pre-trained weights for the two frameworks, and the description notes that Keras and PyTorch weights are included. A third directory, competition/, holds the code for the competition entry, the one incorporated with nnU-Net.

The patch generator lives at the root as infinite_generator_3D.py, which matches the fact that the distortion pipeline is the core of the method and the file name suggests it can produce distortions without a fixed length. The figures/ directory holds the logo, the patch generator diagram, the framework diagram, the results plot, and the two award photographs. The results plot is produced by Matlab code in figures/plotsuperbar.m with helper functions in figures/superbar, credited to Scott Lowe's superbar, which tells you the paper figures were produced in Matlab rather than Python.

There is a requirements.txt and a .gitignore, and the primary language is recorded as Jupyter Notebook, which fits a research release that ships notebooks for demonstration alongside the framework code.

The framework diagram and the patch generator diagram are the two figures to look at first if you want to understand the method without reading the paper, because they carry the pipeline and the distortion types in one image each.

## The dependency set is the first thing that will stop you

The requirements file is where the age of this release shows most clearly, and you should read it before planning anything.

The file is commented as usable to create an environment, and the suggested route is conda:

```bash
# $ conda create --name <env> --file <this file>
```

The packages are tqdm, sklearn, scikit-image, simpleitk, scipy and matplotlib, which are current enough to be unremarkable. The deep learning pins are the problem:

```bash
tensorflow-gpu==1.14.0
keras==2.2.4
torch==1.3.1
torchsummary
```

TensorFlow 1.14 predates the 2.0 release by years, and it is the GPU build at that version. Keras 2.2.4 is the version that ships separately from TensorFlow rather than the modern integrated one. PyTorch 1.3.1 is from the same period.

The consequence is practical rather than theoretical. The pre-trained weights are the valuable artifact, and loading them requires an environment matching the framework generation they were trained in, or a port. The README does not document a TensorFlow 2 or PyTorch 2 migration path, and there is no compatibility note about the modern versions of these libraries.

Note also that simpleitk and scikit-image are in the list, which tells you the data pipeline handles medical volume formats directly rather than expecting pre-converted arrays, and that the plotting and numeric stack is the older scientific Python set.

## The results the repository claims, and the caveat attached to one of them

Three results are listed as the major outcomes, and they are worth reading as claims with a specific scope.

First, Models Genesis outperform 3D models trained from scratch. Second, Models Genesis top any 2D approaches, including ImageNet models and degraded 2D Models Genesis. Third, Models Genesis in 2D offer performances equivalent to supervised pre-trained models.

The third one is the most interesting claim to reason about, because it says that a 3D self-supervised model, applied in 2D, matches the performance of models that had actual labels. That is the argument the method needs to make in order to be worth using, and it is a claim about 2D slices rather than about 3D volumes.

There is also a qualification that is easy to miss. The note says that learning from scratch simply in 3D may not necessarily yield performance better than ImageNet-based transfer learning in 2D. In other words, the comparison is 3D self-supervised against 2D supervised, and the honest reading is that a well-chosen 2D ImageNet model is still a strong baseline, not that the 3D route wins in every case.

The competition claim is the one that needs the most care. Models Genesis, incorporated with nnU-Net, ranked first in liver and tumour and hippocampus segmentation on the Medical Segmentation Decathlon as of 2021. The parenthetical note adds that the live leaderboard now loads results client-side and may not display historical standings, so the number cannot be checked against the current page. Treat it as a 2021 result reported by the authors rather than a live ranking.

The project is described as a primary source of transfer learning for 3D medical imaging, particularly where annotated data is limited, and that is the case the framework is designed for.

## Licence, maintenance and what to verify before you build on the weights

The licence is the first thing to check, because it is the one piece of information the repository does not spell out. GitHub records it as a custom licence that it cannot classify, with a LICENSE file at the root as the authoritative text, so read that file before you use the weights or the code in anything commercial. A research release of this kind is often licensed for non-commercial research use, and the exact terms are in that file rather than in the README.

On maintenance, the facts are these. The last push was on 2026-08-25, the repository is not archived, and there are no GitHub releases. So the code is still being touched, and your version of record is a commit rather than a tagged artefact.

The acknowledgements record that the research was supported partially by ASU and Mayo Clinic through a Seed Grant and an Innovat, with the sentence cut off in the visible text. The institutional backing explains how a 2019 project is still being maintained years later, and it also means the project is not a commercial product with a support commitment.

What to verify before you build on this. Check the licence file, since everything downstream depends on it. Check that the framework generation of the weights you need matches what your environment can load, and if not, budget for a port. Read the paper, the PDF is linked from the repository, and the homepage field points to the MedIA publication PDF. And if you are relying on the leaderboard claim, check the leaderboard yourself rather than the README summary, because the README already warns you that the page no longer shows historical standings.

For the unlabelled data question specifically, the framework gives you a starting representation, and the fine-tuning stage on your own annotated task is where your data still has to come from. The savings are in the pre-training step, not in the labels for your target task.

## Conclusion

Adopt Models Genesis if you are building 3D segmentation or classification on CT or MRI and you have volumes but not annotations, since the pre-trained weights remove the labelling cost from the first step. Do not adopt it if your environment is on a modern TensorFlow 2 or PyTorch 2 stack, because the declared dependencies are pinned to TensorFlow 1.14, Keras 2.2.4 and PyTorch 1.3.1, and expect to do the porting yourself. Before you build on the weights, read the LICENSE file yourself, since GitHub records the licence as a custom one it cannot classify, and check the paper PDF linked from the repository rather than relying on the leaderboard claim, which the README itself flags as no longer displaying historical standings.

## FAQ

### What self-supervised tasks does Models Genesis use for pre-training?

It uses image restoration proxy tasks: non-linear intensity transformation, local pixel shuffling, out-painting and in-painting. The model learns anatomy by reconstructing deliberately distorted volumes, which places it in the generative or restorative family rather than the contrastive one.

### Which frameworks does Models Genesis provide code and weights for?

Keras and PyTorch, in separate keras/ and pytorch/ directories, with pre-trained weights for both included. A third directory, competition/, holds the nnU-Net competition entry.

### What does the Models Genesis requirements file pin?

The deep learning dependencies are pinned to tensorflow-gpu==1.14.0, keras==2.2.4 and torch==1.3.1, with torchsummary alongside, and the scientific stack of tqdm, sklearn, scikit-image, simpleitk, scipy and matplotlib.

### Which papers should I cite if I use the Models Genesis code or weights?

The MICCAI 2019 paper Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis and the Medical Image Analysis journal article Models Genesis, volume 67, pages 101840, in 2021. The repository provides BibTeX for both.

## Sources

- [Issues](https://github.com/MrGiovanni/ModelsGenesis/issues)
- [MrGiovanni/ModelsGenesis on GitHub](https://github.com/MrGiovanni/ModelsGenesis)
- [Project website](https://www.cs.jhu.edu/~zongwei/publication/zhou2020models.pdf)
- [README](https://github.com/MrGiovanni/ModelsGenesis/blob/master/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/mrgiovanni-modelsgenesis