Kornia: A Differentiable Vision Library That Wants to Be the Spec, Not Just the Toolkit

The gap Kornia fills: vision ops that sit inside the autograd graph

Most image processing code is written to run before the model. You resize, blur, convert colour spaces, then hand a tensor to PyTorch. That works until a loss depends on the transform itself, or until you want gradients to flow back through a warp, a filter or a photometric adjustment. Kornia's premise is that these operations should be torch modules: differentiable, batched, and executable on GPU alongside the rest of the network. The README describes it as a differentiable computer vision library built on top of PyTorch, with batch transformations, auto-differentiation and GPU acceleration as the three properties it leans on. The audience follows from that. If you are training models and your pipeline already lives in PyTorch tensors, Kornia removes the boundary between preprocessing and the graph. If your work is CPU image manipulation with no training loop, the differentiability buys you nothing and the PyTorch dependency is pure overhead.

What is actually in the library: operators, augmentation containers, and model wrappers

The README splits the library into three named components. The first is differentiable image processing: filters such as Gaussian, Sobel, median and box blur; transformations including affine, homography and perspective; enhancements such as histogram equalisation, CLAHE and gamma correction; edge detection with Canny, Laplacian and Sobel. The second is augmentation, organised around pipeline containers: AugmentationSequential, PatchSequential and VideoSequential, plus automatic policies named AutoAugment, RandAugment and TrivialAugment. The third is pre-trained AI models, listed as YuNet for face detection, LoFTR and LightGlue for feature matching, DISK, DeDoDe and SOLD2 as descriptors, SAM for segmentation, and MobileViT and VisionTransformer for classification. The README claims 1,200+ public operators, including camera models and epipolar geometry, minimal solvers and RANSAC, feature detection, description and matching, morphology, and image losses and metrics. That list is a catalogue, not a compatibility statement. The practical reading is that Kornia spans two very different kinds of code under one import name: deterministic geometry and filtering, which you can reason about mathematically, and learned model wrappers, whose behaviour depends on downloaded weights and the upstream research implementation. Treat them as separate adoption decisions.

The stated direction: reference implementation and executable specification

The most consequential line in the README is the direction statement, which says Kornia is becoming the reference implementation and executable specification for differentiable computer vision and geometry in the PyTorch ecosystem, and that this means explicit conventions, conformance tests and honest benchmarks over API growth. That is a claim about governance, not features, and it has a concrete downside for users. A library that prioritises conformance over API growth will rename things, tighten conventions and deprecate loose behaviour rather than keep adding surface area. The v0.9.0rc1 release is titled compile-first, deployable augmentations, which suggests the augmentation path is being reworked for compiled execution and deployment rather than simply extended, and v0.9.0 followed in October 2026. If you adopt Kornia, budget for the possibility that the operator you import today is spelled differently or lives in a different module after a minor version.

Getting it running: install, import, and the container you pick

The package is installed from PyPI with pip install kornia. The project publishes to PyPI (the README carries a PyPI version badge) and the documentation lives at kornia.readthedocs.io, which is where the module paths and API signatures should be checked rather than inferred from the README's prose. What the README does name explicitly, and what therefore matters at configuration time, are the augmentation containers. AugmentationSequential, PatchSequential and VideoSequential are distinct entry points, and the choice is not cosmetic: they encode different assumptions about what a sample is. A single image, a set of image patches, and a video clip are not interchangeable inputs, and the container you instantiate determines the shape your dataset must return. The automatic augmentation policies (AutoAugment, RandAugment, TrivialAugment) are the other named configuration surface. Kornia is a library rather than a service, so any setup instruction beyond install and import has to come from the docs, and you should read the docs for the specific version you pin.

Where Kornia is the wrong tool

The clearest limitation is structural: Kornia is a PyTorch library. Every operator returns or consumes tensors, and the value proposition depends on autograd and GPU execution. If your pipeline is C++, or runs in a serving stack that does not carry PyTorch, or processes single images on CPU where the overhead of tensor construction dominates the operation, Kornia adds a dependency without returning the differentiability you are paying for. A second limitation is the breadth itself. The README claims 1,200+ public operators across camera models, epipolar geometry, solvers, feature detection, morphology, colour and image losses and metrics. Breadth at that scale means uneven depth. A third issue is the release state. The most recent release is v0.9.0 (23 October 2026), which followed the v0.9.0rc1 release candidate and v0.8.3. The compile-first, deployable augmentation work only recently reached a stable tag, so expect that area to keep moving.

How it differs from OpenCV, and why the difference is not just differentiability

The obvious comparison is OpenCV, and the difference is not that one is differentiable and the other is not. OpenCV is a mature C++ library with Python bindings, decades of accumulated operator coverage, and a design centre of single-image, CPU-first processing. Kornia's design centre is a batch of tensors on a device, with gradients attached. That changes the API shape: OpenCV functions take and return arrays and mutate in place; Kornia's operators are torch modules that participate in a graph. It also changes the failure modes. In OpenCV, a wrong argument produces an exception or a wrong image. In Kornia, an operator that is differentiable but numerically unstable produces a wrong gradient, which is harder to notice because the forward output can look correct. The honest framing is that these are not substitutes. If you need a homography estimate inside a training loop so a loss can backpropagate through it, OpenCV is not an option and Kornia is. If you need to deskew a scanned document in a batch job, Kornia is the heavier choice.

Maintenance, release cadence and the Apache-2.0 terms

The release history shows v0.8.3 in May 2026, v0.9.0rc1 in July 2026 and v0.9.0 in October 2026, with the repository's last push on 2 November 2026. The stable v0.9.0 has now followed the release candidate. For a library you import rather than run, that cadence is manageable: pin a version, read the changelog before bumping, and expect the augmentation API to be the area most likely to move given the v0.9.0rc1 title. The upgrade cost is concentrated in the augmentation containers and in any operator touched by the conformance work. On licensing, Kornia is Apache-2.0, which permits commercial use and modification and includes a patent grant, with the usual obligations around preserving notices and stating changes. Apache-2.0 is permissive and does not impose copyleft on your code. What the licence does not settle is the provenance of pre-trained weights for the model wrappers listed in the README (YuNet, LoFTR, LightGlue, DISK, DeDoDe, SOLD2, SAM, MobileViT, VisionTransformer). Those may carry their own terms from the original authors, and the repository's licence does not automatically cover them. This is not legal advice; check the weight source for any model you ship.

Editorial conclusion

Adopt Kornia if you already train in PyTorch and need image operators, geometric transforms or augmentation inside the autograd graph, rather than as a preprocessing step outside it. Do not adopt it as a general image-processing replacement for OpenCV in a non-PyTorch pipeline, and do not assume the AI model wrappers (LoFTR, LightGlue, SAM, YuNet) carry the same stability guarantees as the classical operators. Before committing, verify three things against your own version: that the specific operator you need exists in the module path you expect, that the augmentation container you pick (AugmentationSequential, PatchSequential, VideoSequential) matches your input layout, and that you can pin a release rather than tracking main, given that v0.9.0 only reached a stable release in October 2026.

Frequently asked questions

Official sources

1. kornia/kornia on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases