sniklaus/3d-ken-burns: A Reference Implementation With a CUDA Dependency and a Licence You Must Read

The Single-Image Camera Move Problem

A Ken Burns effect is a pan and zoom across a still photograph. Done in two dimensions, it looks flat because every pixel moves at the same rate. The paper this repository implements, cited in the README as 3D Ken Burns Effect from a Single Image, targets the depth cue instead: the camera moves through a scene reconstructed from one image, so nearer objects shift more than distant ones. The README describes the result as animating a still image with a virtual camera scan and zoom subject to motion parallax. The intended user is someone building a video pipeline who has a source photograph and no multi-view capture, no stereo pair, and no lidar. The repository calls itself a reference implementation, which is a meaningful label here: it exists to demonstrate the method in PyTorch, not to be a packaged library with a stable API. If you want a two-dimensional pan and zoom, you do not need this project at all, and the README points to pierlj/ken-burns-effect as related work in that space.

Depth Estimation, Camera Path and Where CuPy Enters

The pipeline has two visible stages. The first produces a depth map from the single input image. The second uses that depth to drive a virtual camera along a path, generating the frames that become the video. The README splits these into separate scripts, and that split is the most important architectural fact in the repository. The README states plainly that depthestim.py does not perform the depth adjustment and links to issue #22 for information on how to add it. In other words, the raw depth estimate is not the depth the effect consumes. Anyone reading only the top-level autozoom.py command will not see this gap. The second architectural fact is the compute substrate. Several functions are implemented in CUDA using CuPy, and the README gives that as the reason CuPy is a required dependency rather than an optional accelerator. The CUDA_HOME environment variable must be configured. This is not a library that degrades to a slower CPU path; the CuPy dependency is stated as required, not conditional.

Getting It Running: Four Commands and Two Install Steps

The setup section lists two installs. CuPy comes from pip install cupy or from one of the binary packages linked in the CuPy documentation, and the README advises matching the binary package to your environment. moviepy is needed to generate video results. The usage section gives four entry points. The fully automatic run is python autozoom.py --in ./images/doublestrike.jpg --out ./autozoom.mp4. The manual path is python interface.py, which serves an interface at http://localhost:8080/ where you load an image with a button in the bottom right corner. The README warns that loading an image and saving the result involve background processing and asks for patience. The raw depth script is python depthestim.py --in ./images/doublestrike.jpg --out ./depthestim.npy, writing a NumPy array. Verification is python benchmark-ibims.py or python benchmark-nyu.py, which the README describes as a way to confirm the provided implementation runs as expected. If you have no suitable environment, the README lists three third-party Colab notebooks, by Arnaldo Gabriel, Vlad Alex and Ahmed Harmouche, none of which are maintained by the repository author.

The Depth Adjustment Gap and the GPU Requirement

Two constraints deserve to be stated before you plan any integration. The first is the depth adjustment gap described above. The README points to an issue rather than a script, so wiring the adjustment into your own pipeline is work you own, and the repository does not present it as a supported step. The second is hardware. With CuPy as a required dependency and CUDA_HOME as a prerequisite, a machine without a working CUDA installation is not a supported target. The Colab notebooks are the README's own answer for people without a suitable environment, which is an implicit acknowledgement that local setup is a hurdle. There is a third, softer limitation: the README gives no parameter documentation for the camera path in autozoom.py, and the manual interface at port 8080 is described in terms of clicking rather than configuration keys. If you need scripted, reproducible camera paths rather than hand-tuned ones, the material here does not show how to express that.

Dataset Size, Licence Split and the NOASSERTION Problem

The training and evaluation data is not in the repository. It is offered as per-scene archives of color, depth and normal data, and the sizes are large. The city scene is the smallest listed, at 0.8 GB for flying color and 0.2 GB for flying depth. The chill scene is the largest, at 5.4 GB for flying color and 10.8 GB for flying normal. A full download across scenes and modes runs into tens of gigabytes, and the normal maps are consistently the heaviest component. The licensing is split and both halves matter. The dataset is licensed under CC BY-NC-SA 4.0 and the README states it may only be used for non-commercial purposes, with the LICENSE file holding the detail. The repository itself is reported as NOASSERTION, which means the licence could not be automatically identified from the repository contents. That is not the same as having no licence, and it is not the same as having a permissive one. It means you cannot assume terms. Reading the LICENSE file yourself is the only way to know what applies to the code, and none of this is legal advice.

Where a Two-Dimensional Pan and Zoom Is the Better Answer

The real alternative is the flat Ken Burns effect, and the difference is not quality but input assumptions. A two-dimensional pan and zoom needs no depth model, no CUDA, and no CuPy. It scales a crop window across the source image and every pixel moves together. It produces no parallax, so a portrait against a distant background will not separate, but it also cannot produce the artefacts that a wrong depth map produces: halos at object boundaries, foreground edges that bend, and background regions that slide at the wrong rate. The README links pierlj/ken-burns-effect as related work, which is the natural place to look if that trade-off suits you. The choice comes down to whether your source images have clear depth structure and whether you can tolerate depth errors. For a photograph with a strong foreground subject and a receding background, the parallax is worth the setup cost. For a flat-lay, a texture, or a document scan, a depth model has nothing useful to estimate and a two-dimensional move is both cheaper and more predictable.

Maintenance Cost and What to Check Before You Commit

The repository is not archived and the last push recorded is 2026-06-01, so it is not abandoned. There are no releases retrieved, which means there is no versioned artifact to pin against; you would be tracking the master branch. That raises the cost of every upgrade, because there is no changelog to read. Your ongoing maintenance surface is the CUDA and CuPy pairing, which is the part most likely to break when a driver or toolkit changes, plus moviepy for the encoding step. The dataset adds a storage cost and a licence obligation that travels with any derived model or output you distribute. If you proceed, the first thing to verify is that the LICENSE file resolves the NOASSERTION status to terms you can accept for your intended use. The second is that benchmark-ibims.py or benchmark-nyu.py completes on your hardware, since that is the repository's own stated way to confirm the implementation runs as expected. The third is that you have a plan for the depth adjustment step, because depthestim.py leaves it to you and the README only points at issue #22.

Editorial conclusion

Adopt this if you have a CUDA workstation, you are comfortable installing CuPy from a binary package matched to your driver, and your use is non-commercial or you have separately cleared the licensing. Do not adopt it if you need a CPU-only or permissively licensed component, or if you expect the depth adjustment step to be part of the pipeline, because depthestim.py explicitly does not perform it. Before you build anything on top, verify three things: that the repository's LICENSE file resolves the NOASSERTION status to terms you can accept, that the dataset's CC BY-NC-SA 4.0 restriction is compatible with your distribution, and that benchmark-ibims.py or benchmark-nyu.py reproduces the expected results on your own GPU.

Official sources

1. Issues
2. README
3. sniklaus/3d-ken-burns on GitHub