Connect differentiable image geometry to PyTorch training pipelines
This library allows image rotation and blurring to continue without interruption during model training. Operating on PyTorch, it provides image processing and geometry algorithms in a differentiable form, enabling simultaneous use of bulk processing on GPUs and automatic differentiation.
Strengthening the geometry core and shifting direction
The project is currently focusing on establishing itself as the standard implementation of differentiable computer vision and geometry within the PyTorch ecosystem, rather than expanding its API. The top priority is increasing core reliability through explicit conventions, conformance tests, and honest benchmarks. Model additions are currently frozen, and maintenance resources are being directed toward ensuring the accuracy and stability of geometric operations.
Handling diverse parameters in a single batch
Unlike existing libraries that apply only one parameter per call, this tool can apply different values to each sample within a batch.
- Batched minimal solvers When estimating fundamental or essential matrices, it processes multiple samples simultaneously and returns all real solutions. The RANSAC algorithm also samples and scores hypotheses on the device.
- Differentiable geometry It maintains gradients when converting depth information to 3D coordinates or correcting camera model distortion. Gradients can be calculated with respect to depth, pose, and camera parameters.
- GPU-based local features It runs various feature detectors and matchers, such as SIFT, DISK, and LoFTR, on the GPU. It supports mutual nearest neighbor and ratio test matchers.
Performance and precision support status
According to benchmarks run with a batch size of 32 in an RTX 4090 environment, many filter and transformation operations are 10 to 70 times faster than CPU-based OpenCV or scikit-image.
- Half-precision support It maintains a list of known failures for float16 and bfloat16 in CPU environments. Half-precision support for CUDA and MPS is not currently verified in CI, and CPU results do not guarantee GPU support.
- Compilation and export Most operation settings are compatible with
torch.compile, and over 1,000 operations can be exported to ONNX. Export support for each operation can be checked on the official page.
Pre-application checks
This tool is suitable for deep learning pipelines that require complex geometric transformations or bulk image processing. If you only need simple fixed-parameter transformations, PyTorch native features or torchvision may be sufficient. Use pip install kornia for installation, and note that additional packages are required for ONNX or Stable Diffusion-related features. If you plan to use half-precision operations, you must verify the support scope in both CPU and CUDA environments.
