Graphormer: a Transformer backbone for molecular property prediction, and the limits of its own README

The gap Graphormer was built to close

Most graph neural networks for molecules hard-code structural priors: message passing over bonds, hand-designed edge features, fixed neighbourhood aggregation. Graphormer's premise, stated in its NeurIPS 2021 paper title, is that a plain Transformer can do this job if you give it the right encoding of graph structure. The package that follows is aimed at researchers who want to train that backbone on molecular property regression, in particular the quantum-chemistry benchmarks that dominate the field. The README names the intended applications directly: material discovery and drug discovery, framed as AI for molecule science. The repository topics list ai4science, deep-learning, graph, molecule-simulation and transformer, which is an accurate summary of the scope. It is not a cheminformatics toolkit. There is no conformer generation, no SMILES canonicalisation, no docking. It is a model and a training harness for graph-level prediction, and it assumes you bring your own featurisation or use one of the dataset interfaces it wraps.

How the Transformer sees a graph: the mechanism in the code

The mechanism is the part worth understanding before you install anything. Standard self-attention has no notion of distance or connectivity, so Graphormer injects graph structure through the attention computation itself rather than through the input features alone. The NeurIPS paper describes three encodings: a centrality encoding that adds a learned vector to each node based on its degree, a spatial encoding that biases attention scores according to the shortest path distance between node pairs, and an edge encoding that folds bond features into the attention bias. The effect is that two atoms three bonds apart attend to each other differently than two atoms one bond apart, without any message passing loop. The package sits on top of fairseq, which the README lists as a supported backbone, so training runs through fairseq's task and criterion abstractions rather than a bespoke trainer. Data can arrive through PyG, DGL, OGB or OCP interfaces, all named in the v2.0 highlights. That layering matters for debugging: when something fails, the fault may be in your dataset wrapper, in fairseq's batching, or in the model, and the README does not help you tell them apart.

Installation is one script, and that is the whole story

The README's installation section is short to the point of being unhelpful. Under Setup with Conda it gives exactly one command: bash install.sh. There is no listing of what that script does, no pinned CUDA version, no PyTorch version, no Python version floor, and no mention of GPU requirements anywhere in the supplied text. The README does point to a readthedocs site generated from the repository and says it contains instructions for getting started, training new models, and extending Graphormer with new model types and tasks, plus an examples directory showing command line usage of common tasks. Treat that documentation as the real installation guide and the README as a pointer to it. If you are evaluating the project, the honest first step is to open install.sh and read it before running it, because a single shell script that silently resolves a deep learning stack is the most likely place for an environment mismatch on a shared cluster.

Pretrained weights are split between the repo and a commercial service

This is the most consequential detail in the README, and it is easy to skim past. A line set in large type states that advanced pre-trained versions of Graphormer are available exclusively on Azure Quantum Elements, with a link to Microsoft's quantum product page. The open repository offers pretrained models on PCQM4M and PCQM4Mv2, with the README saying more are coming. So the free tier of pretrained weights covers two quantum-chemistry datasets, and anything described as advanced sits behind a commercial product. For an academic group benchmarking on PCQM4Mv2, that is fine. For a team hoping to fine-tune a strong general molecular checkpoint out of the box, the open repository may not contain what you need. The README does not specify checkpoint file sizes, download URLs, or the exact fine-tuning entry points, so verify availability before you plan a project around it.

Release cadence and what the news section tells you

The README's What's New section ends on 03/10/2022, with a technical report on improved benchmarks for PCQM4M and the Open Catalyst Project. The tagged releases are dig-v1.0, Distributional Graphormer, from 2024-04-03, preceded by a pre-release on the same day. The repository's last push is 2026-06-12, so commits continue even though the README's own history does not. That combination is common in research code: the paper artifact is frozen, while maintenance happens in smaller commits that never reach the changelog. The practical consequence is that you cannot infer the state of the code from the release list. You have to read the commit history and the readthedocs build. A project whose documentation site is generated from the repository will drift out of sync with a README that has not been touched in four years, and the README is already the thinner of the two.

Where Graphormer is the wrong tool

Graphormer's spatial encoding depends on shortest path distances between node pairs, which is quadratic in the number of nodes before you even reach the attention matrix. For small molecules this is irrelevant. For a protein with thousands of residues, or a polymer or crystal supercell, the cost of computing and storing those pairwise distances grows faster than the attention itself. The README presents the package as general-purpose for molecule modeling, and it does support the Open Catalyst Project datasets, but nothing in the supplied material claims it scales to very large graphs, and the encoding design argues against assuming it does. A second limitation is the dependency surface. Supporting PyG, DGL, OGB, OCP and fairseq simultaneously means version conflicts are likely, and the one-line install script gives you no way to pin a subset. If you only need message passing on small molecules, a plain PyG model will be easier to install, easier to debug, and easier to keep running.

The alternative approach: message passing instead of attention bias

The direct contrast is with a message-passing neural network built in PyTorch Geometric, the same library Graphormer wraps for data loading. A message-passing model propagates information along bonds for a fixed number of hops, so its receptive field is bounded by depth and its cost scales with edges rather than with all node pairs. Graphormer instead lets every atom attend to every other atom in one layer, with distance encoded as a bias, so a single layer can in principle relate distant parts of a molecule. The trade-off is compute and memory against receptive field. For a dataset of small molecules where the relevant chemistry is local, the message-passing model is the cheaper and more predictable choice. For a benchmark where long-range interactions matter and the graphs are modest in size, which is the setting the PCQM4M results come from, the attention-bias approach is the reason the project exists. Choosing between them is a question about your graph sizes, not about which model is newer.

Licence, contributions and the cost of staying current

Graphormer is MIT-licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is the permissive end of the spectrum and imposes little on adopters. Two caveats come from the repository's own governance text rather than the licence. Contributions require a Contributor License Agreement, with a CLA bot decorating pull requests, so if you intend to upstream fixes you should expect that process. And the trademark section states that use of Microsoft trademarks or logos in modified versions must not cause confusion or imply Microsoft sponsorship, which matters if you fork and redistribute under a name that echoes the original. On maintenance cost, the honest reading is that this is research code with a paper artifact frozen in 2022 and a 2024 release tag, and that the documentation site is where current instructions live. Budget time for reading install.sh and the readthedocs pages rather than expecting the README to carry you, and check the examples directory for a task close to yours before writing a dataset wrapper from scratch.

Editorial conclusion

Adopt Graphormer if you are a research group or ML engineer who already runs PyTorch and Conda, needs a graph Transformer backbone with OGB and OCP dataset plumbing, and is willing to read the readthedocs site rather than the README. Do not adopt it if you need a maintained, frequently released library with a stable API: the last tagged release is Distributional Graphormer from April 2024, the README's own news section stops in March 2022, and the strongest pretrained checkpoints are described as available exclusively on Azure Quantum Elements rather than in this repository. Before committing, verify three things yourself: that bash install.sh completes on your CUDA and PyTorch combination, which pretrained checkpoints are actually downloadable from the repo versus only referenced, and whether the OCP or OGB interface you need is covered by the examples directory or requires writing your own dataset class.

Official sources

1. Issues
2. License: MIT
3. microsoft/Graphormer on GitHub
4. README
5. Releases