REINVENT4: Reinforcement Learning for Molecular Design, Driven by a TOML File
AI molecular design tool for de novo design, scaffold hopping, R-group replacement, linker design and molecule optimization.
At a glance
- What is it?
- REINVENT4 is a command line molecular design tool from AstraZeneca that generates and optimizes small molecules against a user-defined multi-component score. The hard part is not running it, it is writing a score that says what you actually want.
- Who is it for?
- Adopt REINVENT4 if you have a scoring function you trust and a Linux machine with a GPU, and you are willing to write TOML rather than click through a GUI. Do not adopt it if you want a graphical interface, if you need Windows support you can rely on, or if you cannot yet express your design goal as a numeric score.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 14 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The scoring function is the product
Most generative chemistry tools are judged on the molecules they produce. REINVENT4 is better understood as a system for turning a property profile into a reward signal, then letting reinforcement learning search against it. The README states the tool uses an RL algorithm to generate molecules compliant with a user defined property profile defined as a multi-component score. That sentence is the whole design. The model is not the interesting part. The score is.
This matters for who should use it. A medicinal chemist who can articulate a target profile as weighted components (a docking term, a physicochemical filter, a similarity constraint, a synthetic accessibility penalty) gets a search loop that optimizes exactly that profile. A chemist who cannot yet do that gets a tool that will happily optimize something else. The failure mode is not a crash. It is a plausible-looking set of molecules that satisfies the score and not the intent.
The stated task list is broad: de novo design, scaffold hopping, R-group replacement, linker design, and molecule optimization. These are not separate programs. They are run modes selected in the configuration file, which is why the README tells you to choose a model and the appropriate run mode depending on the research problem. Transfer learning is the other lever: it can create or pre-train a model that generates molecules closer to a set of input molecules.
How a run actually flows
The architecture visible from the repository is a pipeline with four stages. A prior model supplies the initial generative distribution. A scoring function evaluates sampled molecules. An RL algorithm updates the model toward higher-scoring output. A configuration file ties the three together.
The prior is the starting point. Public prior models are hosted on Zenodo, and internal priors can be referenced with a dot notation documented in reinvent/prior_registry.py. That dot notation is worth knowing about because it means you can point at a bundled model without hardcoding a path, while external priors need a real file path.
The scoring components are where the CPU time goes. The README notes that for reinforcement learning a GPU is less important because most scoring components run on the CPU. That is a useful correction to the usual assumption that generative chemistry is GPU-bound. If your score calls docking or a descriptor calculation, the bottleneck is that calculation, not the network forward pass.
Configuration is TOML by default, with JSON and YAML supported. TOML was chosen because the project considers it more user friendly, and the CLI lets you force the format with --config-format toml, json or yaml if extension detection is not what you want. Sample configuration files for all run modes live in configs/, alongside several .md files with instructions. Those markdown files are effectively the real manual, since the README defers to them.
Getting it running: conda, uv, and the processor-type argument
Installation has two paths. The conda path is the documented default. Clone the repository, create an environment on Python 3.11 or later, then run the install script with your processor type as a positional argument:
python install.py cu126
Substitute rocm6.4, xpu, mac or cpu as appropriate. The README is explicit that the toolkit version must be compatible with your hardware driver, so a CUDA build paired with an older NVIDIA driver is a setup failure rather than a runtime error you can debug later. Optional dependency groups are selected with -d: openeye for ROCS (you must obtain your own license), chemprop1 for Chemprop v1, isim for similarity tracking in TensorBoard, or none. The default installs all dependencies.
The uv path is labelled experimental. From the repository root, uv sync installs core dependencies, uv sync --extra isim adds similarity tracking, and uv sync --extra all adds OpenEye ROCS. The PyTorch index for CUDA 12.6 is pre-configured in pyproject.toml, which is a convenience if 12.6 matches your driver and an obstacle if it does not.
Running is a single command:
reinvent -l sampling.log sampling.toml
The -l flag redirects logging to a file; omit it to get stderr. Useful flags include -d cuda or -d cpu to override the config file, -s for a random seed, --dotenv-filename for scoring component environment setup, and --log-level, which ranges from verbose down to critical. The verbose level prints per-SMILES output after sampling, which is what you want when a score component is returning nonsense and you need to see the raw values.
Where the tool will disappoint you
The platform story is uneven. The README states REINVENT is developed on Linux and that the Linux version is fully validated, while Windows support is described as less well tested and therefore limited. MacOS support exists through PyTorch's MPS backend, with newer Apple chips named. If your team is standardized on Windows, this is not a tool you can assume will work because the README lists it as supported.
Memory is the second constraint. The README suggests about 8 GiB for both CPU main memory and GPU memory is sufficient for many design tasks. That word many is doing real work. Transfer learning and model training are the memory-hungry modes, which is also where the GPU recommendation is strongest. A CPU-only run is possible and the code falls back automatically when no GPU is present, but the README frames that as a performance compromise rather than an equal option.
There is a subtler limitation in the interface itself. REINVENT4 is a command line tool with no homepage and no graphical front end. Every experiment is a TOML file, and file paths in the sample configs must be adjusted to your local installation. That is fine for a computational chemist who wants version-controlled experiment definitions. It is a poor fit for a bench scientist who wants to sketch a molecule and press a button.
Finally, the README is truncated in the material available here, and the notebooks section is cut off mid-word. The tutorials exist, but the extent of their coverage cannot be confirmed from what is shown.
What it is not: a replacement for a docking or QSAR stack
REINVENT4 sits on top of scoring tools rather than replacing them. A conventional virtual screening workflow enumerates a catalog or a combinatorial library and ranks it. REINVENT4 inverts that: it generates candidates and uses the ranking as a training signal, iterating until the generator drifts toward the high-scoring region. The practical difference is that screening can only find what is already in the library, while REINVENT4 can propose structures outside it. The cost is that you no longer have a fixed, auditable candidate list, and the score becomes the specification.
That makes the choice of scoring components the real architectural decision. The optional OpenEye ROCS dependency, which requires a separate license, and the Chemprop integrations point at the intended pattern: wrap an existing predictor as a scoring component and let RL optimize against it. If you already have a validated QSAR model or a docking protocol, REINVENT4 is a search layer above it. If you do not, REINVENT4 will optimize whatever proxy you hand it, including a bad one.
Similarity tracking through the isim extra and TensorBoard is the closest thing to a sanity check built into the workflow. It lets you watch whether generated molecules stay near a reference set as training proceeds. Without that, a run that has collapsed onto a degenerate high-scoring motif can look like success in the log.
Maintenance, releases, and the Apache-2.0 terms
The release cadence visible in the repository is roughly two to three releases a year: v4.6.22 in July 2025, v4.7 in November 2025, v4.8 in June 2026. That is a maintained project, not an abandoned one, and the last push date is consistent with active development. For a tool that wraps PyTorch and cheminformatics libraries, that cadence matters less than the dependency churn underneath it. Expect to reinstall when PyTorch moves, because install.py takes the processor type and toolkit version as arguments rather than resolving them for you.
The licence is Apache-2.0, which permits commercial and academic use and modification, and includes an explicit patent grant. That is a permissive licence, and it is a meaningful difference from copyleft cheminformatics tooling. It does not extend to the optional dependencies: the README notes that OpenEye ROCS requires you to obtain your own license, and that obligation is separate from REINVENT4's own terms. Chemprop carries its own licence, and the README links to a conversion guide between Chemprop v1 and v2, which suggests the v1 path is legacy. This is a description of the stated terms, not legal advice; check the licence files in the repository and in each optional dependency before commercial deployment.
The upgrade cost is mostly configuration drift. Because file paths in the sample configs must be adjusted locally, and because the configs directory ships markdown instructions that change between releases, a working TOML from v4.6 is not guaranteed to run unchanged on v4.8. Pin your environment and re-read the configs/ markdown files when you move versions.
Who should pick this up, and what to check first
The strongest fit is a computational chemistry group on Linux with GPU access, an existing scoring function they trust, and a need to explore chemistry outside a fixed library. Scaffold hopping and linker design in particular are tasks where enumeration is awkward and generation is natural, and those are named run modes with sample configs in the repository. The second fit is a methods group that wants to pre-train a prior on a proprietary compound set via transfer learning and then run RL against a target profile.
The weak fit is anyone who needs a GUI, anyone on Windows without a Linux fallback, and anyone whose design goal is still a sentence rather than a score. For that last group, the honest first step is to write the score down and test it on known molecules before pointing REINVENT4 at it, because the tool will optimize the score you wrote, not the goal you meant.
Two checks are worth doing before a real run. First, confirm your driver and toolkit versions match, since install.py cu126 or rocm6.4 fails at setup when they do not. Second, run a short sampling job with --log-level verbose against your chosen prior from the Zenodo release and read the per-SMILES output. If the prior is not generating plausible chemistry for your target region, no amount of RL will fix it, and you will have learned that in one run rather than one month.
Editorial conclusion
Adopt REINVENT4 if you have a scoring function you trust and a Linux machine with a GPU, and you are willing to write TOML rather than click through a GUI. Do not adopt it if you want a graphical interface, if you need Windows support you can rely on, or if you cannot yet express your design goal as a numeric score. Before committing, verify two things: that your CUDA or ROCm version matches your driver, since install.py takes the processor type as an argument, and that the prior you pick from the Zenodo release actually generates chemistry in the region you care about. That second check costs one sampling run and tells you more than any benchmark table.
Community notes