Kornia: A Differentiable Vision Library That Wants to Be the Spec, Not Just the Toolkit

The gap Kornia fills: vision ops that sit inside the autograd graph

Most image processing code is written to run before the model. You resize, blur, convert colour spaces, then hand a tensor to PyTorch. That works until a loss depends on the transform itself, or until you want gradients to flow back through a warp, a filter or a photometric adjustment. Kornia's premise is that these operations should be torch modules: differentiable, batched, and executable on GPU alongside the rest of the network. The README describes it as a differentiable computer vision library built on top of PyTorch, with batch transformations, auto-differentiation and GPU acceleration as the three properties it leans on. The audience follows from that. If you are training models and your pipeline already lives in PyTorch tensors, Kornia removes the boundary between preprocessing and the graph. If your work is CPU image manipulation with no training loop, the differentiability buys you nothing and the PyTorch dependency is pure overhead.

What is actually in the library: operators, augmentation containers, and model wrappers

The README splits the library into three named components. The first is differentiable image processing: filters such as Gaussian, Sobel, median and box blur; transformations including affine, homography and perspective; enhancements such as histogram equalisation, CLAHE and gamma correction; edge detection with Canny, Laplacian and Sobel. The second is augmentation, organised around pipeline containers: AugmentationSequential, PatchSequential and VideoSequential, plus automatic policies named AutoAugment, RandAugment and TrivialAugment. The third is pre-trained AI models, listed as YuNet for face detection, LoFTR and LightGlue for feature matching, DISK, DeDoDe and SOLD2 as descriptors, SAM for segmentation, and MobileViT and VisionTransformer for classification. A collapsible table in the README claims more than 500 operations in total and groups them by category, including stereo vision, image registration, pose estimation with essential and fundamental matrix estimation and PnP solvers, and optical flow. That table is a catalogue, not a compatibility statement. The practical reading is that Kornia spans two very different kinds of code under one import name: deterministic geometry and filtering, which you can reason about mathematically, and learned model wrappers, whose behaviour depends on downloaded weights and the upstream research implementation. Treat them as separate adoption decisions.

The stated direction: reference implementation and executable specification

The most consequential line in the README is the direction statement, which says Kornia is becoming the reference implementation and executable specification for differentiable computer vision and geometry in the PyTorch ecosystem, and that this means explicit conventions, conformance tests and honest benchmarks over API growth. That is a claim about governance, not features, and it has a concrete downside for users. A library that prioritises conformance over API growth will rename things, tighten conventions and deprecate loose behaviour rather than keep adding surface area. The v0.9.0rc1 release is titled compile-first, deployable augmentations, which suggests the augmentation path is being reworked for compiled execution and deployment rather than simply extended. If you adopt Kornia, budget for the possibility that the operator you import today is spelled differently or lives in a different module after a minor version. The README does not promise API stability across versions, and nothing in the supplied material describes a deprecation policy or a support window.

Getting it running: install, import, and the container you pick

The README does not include an installation block, so the only command that can be stated with confidence from the material is the PyPI package name: pip install kornia. The project publishes to PyPI (the README carries a PyPI version badge) and the documentation lives at kornia.readthedocs.io, which is where the module paths and API signatures should be checked rather than inferred from the README's prose. What the README does name explicitly, and what therefore matters at configuration time, are the augmentation containers. AugmentationSequential, PatchSequential and VideoSequential are distinct entry points, and the choice is not cosmetic: they encode different assumptions about what a sample is. A single image, a set of image patches, and a video clip are not interchangeable inputs, and the container you instantiate determines the shape your dataset must return. The automatic augmentation policies (AutoAugment, RandAugment, TrivialAugment) are the other named configuration surface. Beyond that, the README gives no config keys, no YAML, and no environment variables, because Kornia is a library rather than a service. Any setup instruction beyond install and import has to come from the docs, and you should read the docs for the specific version you pin.

Where Kornia is the wrong tool

The clearest limitation is structural: Kornia is a PyTorch library. Every operator returns or consumes tensors, and the value proposition depends on autograd and GPU execution. If your pipeline is C++, or runs in a serving stack that does not carry PyTorch, or processes single images on CPU where the overhead of tensor construction dominates the operation, Kornia adds a dependency without returning the differentiability you are paying for. A second limitation is the breadth itself. The README claims more than 500 operations across image processing, augmentation, feature detection, geometry, deep learning layers, photometric functions, filtering, colour, stereo, registration, pose and optical flow. Breadth at that scale means uneven depth. The README lists categories; it does not state which operators are covered by the conformance tests the direction statement refers to, and it does not state which are considered stable. A third issue is the release state. The most recent release in the supplied material is v0.9.0rc1, a release candidate, with v0.8.3 before it. A release candidate as the newest artefact means the deployable augmentation work is not yet final, and anyone adopting it should expect to track the stable line instead.

How it differs from OpenCV, and why the difference is not just differentiability

The obvious comparison is OpenCV, and the difference is not that one is differentiable and the other is not. OpenCV is a mature C++ library with Python bindings, decades of accumulated operator coverage, and a design centre of single-image, CPU-first processing. Kornia's design centre is a batch of tensors on a device, with gradients attached. That changes the API shape: OpenCV functions take and return arrays and mutate in place; Kornia's operators are torch modules that participate in a graph. It also changes the failure modes. In OpenCV, a wrong argument produces an exception or a wrong image. In Kornia, an operator that is differentiable but numerically unstable produces a wrong gradient, which is harder to notice because the forward output can look correct. The honest framing is that these are not substitutes. If you need a homography estimate inside a training loop so a loss can backpropagate through it, OpenCV is not an option and Kornia is. If you need to deskew a scanned document in a batch job, Kornia is the heavier choice.

Maintenance, release cadence and the Apache-2.0 terms

The supplied release history shows v0.8.2 in November 2025, v0.8.3 in May 2026, and v0.9.0rc1 in July 2026, with the repository's last push in September 2026. That is a cadence of roughly two releases a year, with the current head of the line being a release candidate. For a library you import rather than run, that cadence is manageable: pin a version, read the changelog before bumping, and expect the augmentation API to be the area most likely to move given the v0.9.0rc1 title. The upgrade cost is concentrated in the augmentation containers and in any operator touched by the conformance work. On licensing, Kornia is Apache-2.0, which permits commercial use and modification and includes a patent grant, with the usual obligations around preserving notices and stating changes. Apache-2.0 is permissive and does not impose copyleft on your code. What the licence does not settle is the provenance of pre-trained weights for the model wrappers listed in the README (YuNet, LoFTR, LightGlue, DISK, DeDoDe, SOLD2, SAM, MobileViT, VisionTransformer). Those may carry their own terms from the original authors, and the repository's licence does not automatically cover them. This is not legal advice; check the weight source for any model you ship.

Editorial conclusion

Adopt Kornia if you already train in PyTorch and need image operators, geometric transforms or augmentation inside the autograd graph, rather than as a preprocessing step outside it. Do not adopt it as a general image-processing replacement for OpenCV in a non-PyTorch pipeline, and do not assume the AI model wrappers (LoFTR, LightGlue, SAM, YuNet) carry the same stability guarantees as the classical operators. Before committing, verify three things against your own version: that the specific operator you need exists in the module path you expect, that the augmentation container you pick (AugmentationSequential, PatchSequential, VideoSequential) matches your input layout, and that you can pin a release rather than tracking main, given that v0.9.0rc1 is a release candidate.

Official sources

1. kornia/kornia on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases