# ptlflow: one PyTorch Lightning harness for a shelf of optical flow models

> ptlflow collects twenty-plus optical flow estimators behind a single PyTorch Lightning setup, so the same training script, the same config format and the same evaluation harness work across RAFT, GMFlow, NeuFlow and the rest. The author is candid that it is under development, and the 0.4 line broke compatibility with 0.3 to move to Lightning 2.

**hmorimitsu/ptlflow** — PyTorch Lightning Optical Flow models, scripts, and pretrained weights.

- Repository: https://github.com/hmorimitsu/ptlflow
- Stars: 555 · Forks: 68
- Language: Python
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/hmorimitsu-ptlflow

## The point is comparability, not the model count

This is a collection of state-of-the-art deep models for estimating optical flow, and the main goal is to provide a unified framework where multiple models can be trained and tested more easily.

That is the right framing. Optical flow is an area with a long tail of published estimators, each with its own paper, its own data augmentation choices, its own evaluation quirks and often its own training schedule. Comparing them properly means reimplementing each one faithfully, and that is where research effort usually goes wrong. Putting them behind one Lightning setup, one config format and one evaluation harness means the comparison is at least consistent in how it is produced.

The provenance matters as much as the code. The work and code from many others are present here, and the author says he tried to make sure everything is properly referenced but asks to be told if something was missed. Each model in the list links to its arXiv paper, so the reference trail is direct.

The caveats are stated in the same paragraph as the ambition. It is still under development, so some things may not work as intended, and the plan is to add more models and keep improving the platform.

The package is named ptlflow, the primary language is Python, the licence is Apache 2.0, and the version in the manifest is 0.4.3 with the development status classified as Alpha. The last push was on 2026-10-18, and the most recent tagged release is v0.4.2 from 2026-03-28.

## A catalogue that keeps growing, release by release

The model list is long, and the What's new section is the better way to read it, because each entry is a paper that became usable inside the framework.

Version 0.4.2 added Flow-Anything, FlowSeek, MEMFOF, ReCoVEr and WAFT, and in the same release fixed compatibility with Lightning below 2.7 and added support for logging to SwanLab. Version 0.4.1 added DPFlow and StreamFlow, along with a 4K read mode for the Spring dataset, missing arguments for the Middlebury-ST and VIPER datasets, and fixes to the RAFT and RAPIDFlow config files.

Version 0.3.2 added MemFlow, NeuFlow, SEA-RAFT and SplatFlow, extended dataset support to TartanAir and Kubric, added ONNX and TensorRT conversion to RAPIDFlow, and fixed the learning rate scheduler when accumulating gradients.

Version 0.3.1 is where the mixed precision work landed. FP16 was enabled in most models, with three exceptions because they have operations that cannot run in FP16: lcv_raft, matchflow and separableflow. FP16 mode was also added to the infer, model_benchmark and validate scripts, and the resize operations were moved to CUDA.

Version 0.3.0 was the large one, with DIP, Flow1D, FlowFormer++, GMFlow+, UniMatch, MatchFlow, MS-RAFT+, RPKNet, SeparableFlow, SKFlow and VideoFlow all arriving at once, alongside compatibility fixes for PyTorch 2.0 and PyTorch Lightning 1.9, a fix for resizing augmentation when the valid mask is sparse, and support for the Middlebury, Monkaa and Spring datasets.

The pattern across those entries is that a model is not just added but made usable: the datasets, the config files, the inference path and the evaluation scripts all have to work for it before the release goes out.

## Version 0.4 broke compatibility to reach Lightning 2

The v0.4.0 entry is the one to read carefully if you are coming from an earlier version, because it is a major update that also introduces breaking changes from the previous 0.3 code.

The entry says the main event is support for Lightning 2, with the parenthetical finally, and then points at a v0.4 upgrade guide. So the move from 0.3 to 0.4 was not a version bump, it was a rebase of the training layer onto a new major release of Lightning, and the code had to change shape with it.

Two features came with that release. Support for YAML config files, with a separate documentation page for the config file format, and a table comparing PTLFlow results with the original papers, which the README describes as a way to check the stability of the included models.

That comparison table is the most valuable artefact in the project for a newcomer, and it is also the one to be sceptical about. A framework that reimplements twenty estimators will not reproduce all of them exactly, and a table that says so per model is the honest way to communicate that. It also means the framework's own numbers are the ones you should compare against each other, rather than the published figures, when you are asking which model is better.

The dependency constraint in the manifest reflects the same transition. The Lightning dependency is pinned as lightning[pytorch-extra]>=2.1,<2.7, and the 0.4.2 release note says compatibility with Lightning below 2.7 was fixed, which is a slightly awkward corner of a range whose lower bound is 2.1.

## Four scripts at the root define the whole workflow

The repository layout is where the framework becomes concrete, and the top level is a set of entry points rather than a package of abstractions.

There is train.py for training, infer.py for running a trained model, validate.py for evaluation, and model_benchmark.py for measuring speed. Then there is compare_paper_results.py and summary_metrics.py for the comparison against published numbers, and plot_results.py for the figures, which was added in version 0.3.1.

Two naming details tell you about the history. model_benchmark.py was called speed_benchmark.py before version 0.3.0, when it was renamed and made to record more metrics, so the benchmark grew from a timing script into something that reports more than time. And the fact that there are separate infer, model_benchmark and validate scripts rather than one, with the FP16 mode added to all three in 0.3.1, shows that the same model configuration is expected to be driven through all of them.

Inside the package directory, ptlflow/, the structure follows the model family convention with a backbones/ directory of adapters, a core/ directory for config, registry and interfaces, a data/ directory for datasets, collators and transforms, an evaluation/ directory for runners and metrics, an inference/ directory for the pipeline with its batching and generation, a models/ directory for builders, heads and processors, a post_training/ directory for the SFT, IRG, recA and UniCot methods, and a serving/ directory for serving APIs. The cli/ directory holds the entry points, and there is an inference/ directory for the pipeline.

The top level also holds a datasets.yaml file and a configs/ directory, which is where the YAML config support from version 0.4 lives.

## The dependency set tells you what the tooling is made of

The runtime dependencies are listed with upper bounds as well as lower ones, which is unusual to see and worth reading, because it tells you the maintainer tracks upstream breakage.

The list is: einops under 0.9, h5py under 3.17, kaleido under 1.3, lightning with the pytorch-extra extra at 2.1 or newer and below 2.7, loguru under 0.8, opencv-python under 4.14, pandas under 3.1, plotly under 6.7, pypng pinned exactly at 0.20220715.0, requests under 2.34, scipy under 1.18, tabulate under 0.11, tensorboard under 2.21, and timm under 1.1.

Several of those are worth pausing on. timm is the model zoo that many of these vision architectures are built on, so the framework leans on it rather than reimplementing backbones. opencv-python is doing image and video decoding for the optical flow datasets. h5py is for the large dataset formats that optical flow benchmarks ship in. plotly with kaleido is the plotting and static image export pair, which is what plot_results.py uses. pypng is pinned to an exact version because it is used for reading PNG flow files, which are stored as a colour image plus a validity mask, and that format is easy to get wrong.

The project also requires Python 3.10 or newer and below 3.15, and the classifiers list 3.10 through 3.14. The build is setuptools with the package data including markdown, text, licence and font files, and the ttf extension suggests a bundled typeface that travels with the package.

The script and the package are separated in the same way, with the manifest including the ptlflow package and excluding tests from the wheel.

## What it does not promise

Two things are worth being blunt about before you build a research plan on this.

The first is that it is a research platform, not a library. The development status is Alpha, the README says it is still under development and that some things may not work as intended, and the author describes the plan as adding more models and improving the platform rather than stabilising an interface. If you need an optical flow model as a dependency with a version contract, this is the wrong shape of thing.

The second is the half-precision caveat, which is easy to miss and bites exactly the people who care about speed. FP16 is enabled in most models, but three cannot run in it: lcv_raft, matchflow and separableflow, because they have operations that do not survive the conversion. The fix for the model_benchmark and infer paths is that FP16 mode is a mode you choose, not a default you inherit, and for those three models it is not available at all.

The third is the version coupling, and it is the practical one. The Lightning layer, the PyTorch layer and the model configurations all move, and the release notes show the maintainer fixing compatibility for Lightning below 2.7 in one release and for PyTorch 2.0 and Lightning 1.9 in another. That is normal work for a research framework that tracks many upstream projects, but it means pinning a version and reading the changelog is part of using it.

What you get in exchange is the thing the project set out to provide: a way to run the same experiment across many estimators without rewriting the experiment each time.

## Conclusion

Adopt ptlflow if you are doing optical flow research and want to compare many estimators under one training and evaluation setup, with the same datasets and the same metrics, instead of maintaining a separate codebase per paper. Do not adopt it if you need a stable library API, since the project is classified as alpha, the author says some things may not work as intended, and the 0.4 release introduced breaking changes from 0.3. Before you start, read the table comparing PTLFlow results with the original papers, because that is the document that tells you whether a given model reproduces its published numbers, and follow the v0.4 upgrade guide if you are coming from 0.3.

## FAQ

### What is ptlflow?

It is a collection of state-of-the-art deep models for estimating optical flow, packaged as a unified framework where multiple models can be trained and tested with the same code. The work and code from many others are included, and the README says the author tried to make sure everything is properly referenced.

### How do I install ptlflow?

The package is ptlflow on PyPI, requires Python 3.10 or newer and below 3.15, and depends on lightning with the pytorch-extra extra between 2.1 and 2.7, along with torch-adjacent libraries including timm, opencv-python, h5py, scipy and plotly. The build is setuptools and the package data includes markdown, text, licence and font files.

### Which models can ptlflow train and evaluate?

The available list includes CCMR, CRAFT, CSFlow, DICL-Flow, DIP, DPFlow, FastFlowNet, Flow1D, Flow-Anything, FlowFormer and more, with each entry linking to its arXiv paper. Recent releases added Flow-Anything, FlowSeek, MEMFOF, ReCoVEr and WAFT in version 0.4.2.

### What changed between ptlflow 0.3 and 0.4?

Version 0.4.0 is a major update that supports Lightning 2 and also introduces breaking changes from the previous 0.3 code, with an upgrade guide published for the transition. It also added YAML config file support and a table comparing PTLFlow results with the original papers.

## Sources

- [hmorimitsu/ptlflow on GitHub](https://github.com/hmorimitsu/ptlflow)
- [Issues](https://github.com/hmorimitsu/ptlflow/issues)
- [License: Apache-2.0](https://github.com/hmorimitsu/ptlflow/blob/main/LICENSE)
- [README](https://github.com/hmorimitsu/ptlflow/blob/main/README.md)
- [Releases](https://github.com/hmorimitsu/ptlflow/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/hmorimitsu-ptlflow