Delphi learns the natural history of disease with a GPT-2 model trained on 400,000 patient trajectories

The model, the paper and the lineage

Delphi is the model from a Nature paper, Learning the natural history of human disease with generative transformers, and the repository holds the code for the modified GPT-2 model used in it along with the training code and the analysis notebooks.

The authors list is long and includes Artem Shmatko, Alexander Wolfgang Jung and Kumar Gaurav as equal contributors, alongside Søren Brunak, Laust Mortensen, Ewan Birney, Tom Fitzgerald and Moritz Gerstung, with affiliations pointing at DKFZ and EMBL.

The implementation is based on Andrej Karpathy's nanoGPT, and that single fact explains most of the repository's shape. There is no model class to learn, no training framework and no configuration system beyond what nanoGPT already provides. The repository root shows the components: train.py for training, model.py for the model, utils.py, plotting.py, configurator.py, a config/ directory with the training configuration, and separate scripts for evaluation, evaluate_delphi.ipynb alongside evaluate_auc.py and shap-agg-eval.py.

The primary language is recorded as Jupyter Notebook, which reflects that the analysis work is done in notebooks rather than in a package.

The paper itself is the point of the project. This is not a general purpose library for health data; it is the reference implementation of a specific published result, and the notebooks exist to reproduce that paper's figures.

Installation and the demo training run

The setup is a clone, a conda environment and a requirements install.

git clone https://github.com/gerstung-lab/Delphi.git
cd Delphi

conda create -n delphi python=3.11
conda activate delphi
pip install -r requirements.txt

The README notes that installing the requirements normally takes a few minutes, which is the PyTorch download rather than anything unusual.

The requirements file pins torch at 2.3.0 and torchvision at 0.18.0, and constrains numpy below 2.0, which matters because the rest of the stack includes umap-learn, shap and statsmodels and those were not all ready for the NumPy 2 transition. The rest are unpinned: matplotlib, seaborn, shap, pandas, umap-learn, tqdm and statsmodels.

Then the demo training run.

python train.py config/train_delphi_demo.py --device=cuda --out_dir=Delphi-2M

To train on a CPU, remove the --device=cuda argument, and the README points at the config/train_delphi_demo.py file for the training configuration details.

The timing figures give a good sense of the scale. Training a demo model takes around 10 minutes on a single GPU. Training the original model took 1 GPU-hour, on an NVIDIA V100 running CentOS 7. Training on an M1 Macbook Pro's MPS takes around 10 hours.

So the original run was cheap by machine learning standards, which is worth noting for anyone planning the work. A Dockerfile is provided for the training and downstream analyses, at containers/Dockerfile.

The data, and why the demo model is not the paper model

Delphi-2M is trained on 400K patient health trajectories from the UK Biobank data, which is available to researchers upon application.

In addition, the project provides a minimal synthetic dataset that was generated by the trained model itself. The synthetic data is broadly statistically similar to the real data while not disclosing any patient information.

The README is emphatic about what that synthetic data is for: this data and model is for coding testing purposes only, and the authors absolutely do not recommend using this model for any type of research, and instead recommend applying for the full model from the UK Biobank.

That distinction matters more than it might appear. A model trained on synthetic data generated by another model tells you about the generator, not about patients. The synthetic set is a fixture for making the pipeline run, and treating it as a scientific result would be a mistake the README pre-empts.

For working with your own data, the data/README.md file describes how to transform it into a Delphi-compatible format.

There are two data preparation paths for the real thing. To reproduce the training data on the UK Biobank Research Analysis Platform, the supplementary/ukb_rap_training.md guide runs data/ukb_real_data/example_ukb_rap_convert_new.ipynb at a total cost under £10, which is a striking figure and makes the reproducibility effort unusually accessible. Alternatively, to convert a raw UK Biobank .tab basket download locally, the data/ukb_simulated_data/example_ukb_to_bin.ipynb file does that.

The labels file at the repository root, delphi_labels_chapters_colours_icd.csv, carries the ICD chapter mapping with colours, which is how the disease events are grouped for the paper's analyses.

The three analysis notebooks and what each one answers

The notebooks are the substance of the paper, and each one answers a different question.

evaluate_delphi.ipynb evaluates Delphi-2M in terms of prediction accuracy compared to an age-sex-based epidemiological baseline, and the calibration of the predicted risks. It also explores the attention mechanism of the model and the structure of the latent space using UMAP of the learned disease embeddings.

That comparison to an age-sex baseline is the important methodological point. A generative model of disease trajectories is only interesting if it beats the trivial predictor that says a person of this age and sex gets this disease. Anything that does not clear that bar is fitting noise.

shap_analysis.ipynb analyses the SHAP values of the model to understand the importance of the input disease events for the model's predictions for a given patient. Then, using SHAP values aggregated over the entire dataset, it identifies the diseases that are most important for each possible disease in the future.

So it goes from per-patient explanation to population-level structure: which prior conditions matter most for each future condition, across the whole dataset rather than for one case.

sampling_trajectories.ipynb analyses the synthetic data generated by Delphi-2M, comparing it to the real data in terms of the distribution of disease events, ages and disease rates. That is the fidelity check on the sampler, and it is where you find out whether the model produces plausible histories or merely plausible-looking sequences.

And data/ukb_simulated_data/example_ukb_to_bin.ipynb converts raw UK Biobank data into the format required for training, which the README describes as the notebook used for that conversion.

Licensing, which differs between the code and the weights

The two licences are different and the split is explicit.

The code in this repository is MIT licensed. The trained Delphi-2M weights, distributed separately via UK Biobank, are under CC BY-NC-ND 4.0, which is non-commercial use only.

The research restriction is stated in the same section: the model and weights are provided for research purposes only and must not be used for clinical decision-making.

For commercial usage, the README directs you to contact DKFZ Innovation Management at [email protected].

And there is a patent. A patent has been filed for the use of generative transformer architectures to model competing risk and timings of diseases, application number PCT/EP2025/065771, with applicants DKFZ and EMBL, and Moritz Gerstung, Artem Shmatko, Tom Fitzgerald, Ewan Birney, Kumar Gaurav and Alex W Jung listed as inventors.

Put together, the picture for an adopter is a permissive code licence, a non-commercial and no-derivatives licence on the trained weights, an explicit ban on clinical use, a patent over the core method, and a commercial route through the institution. That is a coherent arrangement for academic research with a commercial licensing path, and it is worth understanding before you plan any deployment.

The licence badges at the top of the README make the split visible before you read any of this.

Where Delphi is the wrong choice

Four cases argue against it.

The first is a clinical deployment. The README states the model and weights must not be used for clinical decision-making, and there is no version of this that changes. If your goal is a risk score in a patient pathway, this is the wrong artifact and you should be looking at validated epidemiological models rather than a generative transformer trained on retrospective records.

The second is a commercial product. The weights are non-commercial and no-derivatives, the method is patented, and the commercial route runs through DKFZ Innovation Management. Any of those three is a conversation with an institution rather than a dependency you can take.

The third is a team that wants a general health data model. This is one model, one data source, one paper. It is a modified GPT-2 for a specific task with a specific label set, and the training code is nanoGPT rather than a framework you would extend.

The alternative in the same direction is a conventional survival or competing risks model on the same kind of data, which is what the age-sex baseline in the evaluation notebook represents. The difference in approach is that Delphi models the whole trajectory as a sequence, so it can produce multi-event histories and the timing between events, at the cost of a training run, a GPU and a set of weights with research-only terms. For a question that is one hazard at a time, the classical model is easier to validate and easier to defend to a regulator.

Editorial conclusion

Delphi fits a researcher who wants to model disease trajectories with a generative transformer and needs the training code and the analysis notebooks to work from, since both are in the repository and the model is a nanoGPT derivative rather than a framework. It does not fit anyone who needs the trained weights for commercial use, because they are distributed under CC BY-NC-ND 4.0 through UK Biobank, and the paper's results can only be reproduced with access to the real data. Train the demo with `python train.py config/train_delphi_demo.py --device=cuda --out_dir=Delphi-2M`, apply for UK Biobank access to reproduce the paper, and note that the synthetic dataset is for coding tests only.

Frequently asked questions

Official sources

1. gerstung-lab/Delphi on GitHub
2. Issues
3. License: MIT
4. README