A 200fps Panorama Stitcher Built on OpenCV UMat, With Strings Attached

What This Project Actually Is

This repository is a minimal C++ implementation of a specific research paper: Du, Chengyao, et al., 'GPU based parallel optimization for real time panoramic video stitching,' Pattern Recognition Letters, 2020. The README calls it 'a simple version' of that paper. It uses OpenCV's UMat type to keep image data on the GPU and runs a stitching pipeline over four input video streams. The target user is someone who wants to see how the paper's parallelization strategy translates into code. It is not a general-purpose stitcher. The README is explicit: 'This project does not provide a dataset so it cannot be used out of the box.' That sentence is the most honest part of the whole repository.

The Core Mechanism: UMat and Streaming Parallelism

The project's performance claim, over 200fps on a 1080 Ti for four 4K cameras, rests on two things: OpenCV's UMat and a 'clever streaming parallel strategy.' UMat is OpenCV's unified memory container that lets the same code run on CPU or GPU without explicit transfers. The README does not detail the exact pipeline stages, but the example section shows a sequence: stitching, exposure-mask, exposure-mask-refine, apply-mask, final-panorama. That suggests the algorithm computes exposure masks to blend overlapping regions, refines them, and then applies them to produce the final panorama. The parallelism likely comes from overlapping the processing of different frames or different image regions on the GPU. The README gives no further architecture details, so the exact data flow is not fully documented. What is clear is that the entire pipeline is designed around UMat, which means it will only run fast on hardware and OpenCV builds that support that type.

Getting It Running: Build Commands and the Missing Dataset

The Quick Start section gives three commands: mkdir build && cd build, cmake .., make, and then ./image-stitching. That is the entire build procedure. There is no mention of required OpenCV version, CMake minimum, or any dependency beyond OpenCV itself. The README does not list any configuration keys or command-line options. After building, you run the binary, but it will immediately try to read datasets that do not exist in the repository. The README warns that the program 'will never stop' because the author did not implement a termination condition for the dataset. To make it stop, you are directed to a specific commit that modifies the code. So the practical path is: clone, build, read that commit, patch the code, and supply your own four video files. That is a significant barrier for anyone expecting a runnable demo.

Known Issues: The Real FPS Problem and the Non-Exiting Program

The README lists two known issues. The first is that 'Real FPS' is much slower than 'FPS' because imwrite takes a lot of time. In other words, the display or the saving step becomes the bottleneck. If you are benchmarking this project, you must separate the stitching time from the I/O time. The second issue is that stitching never stops. The program runs indefinitely because the author did not plan for a dataset termination condition. The README suggests following a particular commit to modify the code, and notes that the dataset 'datasets/air-4cam-mp4' will generate 149 pictures. That implies the stop condition is tied to a known dataset length, not to a general input. These are not cosmetic flaws. They affect any real use. If you need a stitcher that processes a finite input and exits cleanly, you will have to write that logic yourself.

The Wrong Tool for Production, But a Useful Reference

This project is wrong for anyone who needs a reliable, maintainable panorama stitcher for a product. There is no dataset, no test suite, no documentation beyond the README, and no license file. The last push was in July 2023, and the repository is not archived, but there is no indication of active development. The README ends with 'Welcome to pull requests,' which suggests a hobby-level maintenance model. As a reference implementation, however, it has value. It shows how to structure a UMat-based pipeline for GPU stitching, and it gives a concrete starting point for someone implementing the paper's ideas. The performance claim, if reproducible, is impressive. But you should treat the 200fps figure as a claim from the author, not a verified benchmark. The repository provides no scripts or instructions to reproduce that number.

Alternatives and the Difference in Approach

The main alternative to this project is OpenCV's built-in stitching module, which provides a high-level Stitcher class. That module works on CPU by default and can be slow for high-resolution video. The difference is that OpenCV's stitcher is a general-purpose tool with many algorithms, but it does not target real-time multi-camera video. Another alternative is to use a commercial or academic system like Autopano Video or VideoStitch, which are designed for real-time panorama stitching but are closed-source and not free. Compared to those, this project is open and shows the exact code, but it is far less complete. If you need real-time performance, you will likely need to build on this project's UMat approach rather than use the generic Stitcher. The paper itself is the best reference for the full algorithm, because the code is only a 'simple version.'

Maintenance and License: What You Are Getting Into

The repository has no license file, and the README does not state a license. That is a legal problem if you want to use the code in a commercial product. You cannot assume it is permissive. Contact the author or find a licensed alternative if you need legal clarity. Maintenance is minimal. There is one release, v1.0.0, from the same date as the last push. No issue tracker activity is visible from the README. The code is small, so upgrading to a newer OpenCV version might be easy, but there is no guarantee. The project depends on OpenCV's UMat, which has changed across versions. You should test with your OpenCV build before committing to it. The README's known issues suggest that the author is aware of the rough edges but has not fixed them. That is a strong signal about the project's maturity.

Editorial conclusion

Adopt this project if you are a researcher or engineer prototyping GPU-accelerated panorama stitching and you are comfortable reading source code and adapting it to your own data. Do not adopt it if you need a production-ready stitcher with a dataset, a stop condition, or a maintained API. Before using it, verify that your OpenCV build has UMat support and that your GPU can handle the memory load of four 4K streams. Also check the linked commit about the dataset branch, because the default build will not terminate on its own. The project is a single-purpose demonstration of a specific paper's idea, not a general-purpose library.

Frequently asked questions

Official sources

1. Official README
2. Project repository
3. Release notes