GeoTorch puts a manifold constraint on a PyTorch layer in one line

The constraint goes on the layer, not in the training loop

GeoTorch provides a simple way to perform constrained optimization and optimization on manifolds in PyTorch.

The integration point is the claim that matters. It is compatible out of the box with any optimizer, layer and model implemented in PyTorch without any boilerplate in the training code. Just state the constraints when you construct the model and you are ready to go.

The example shows how little code that takes:

import torch
import torch.nn as nn
import geotorch

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        # One line suffices: Instantiate a linear layer with orthonormal columns
        self.linear = nn.Linear(64, 128)
        geotorch.orthogonal(self.linear, "weight")

        # Works with tensors: Instantiate a CNN with kernels of rank 1
        self.cnn = nn.Conv2d(16, 32, 3)
        geotorch.low_rank(self.cnn, "weight", rank=1)

So the constraint is applied to a named parameter of a layer, after the layer is constructed. A linear layer gets orthonormal columns, a convolution gets kernels of rank 1, and the code that follows is regular PyTorch:

    def forward(self, x):
        # self.linear is orthogonal and every 3x3 kernel in self.cnn is of rank 1

    # Use the model as you would normally do. Everything just works
    model = Model().cuda()

    # Use your optimizer of choice. Any optimizer works out of the box with any parametrization
    optim = torch.optim.Adam(model.parameters(), lr=lr)

The comment on the Adam line is the one to notice. Any optimizer works out of the box with any parametrisation, which is what separates GeoTorch from the usual constrained-optimisation libraries where you have to use a specific solver for each constraint. The trade is that the parametrisation is what gets optimised, not the constrained quantity, and that is where the library's subtleties live.

The constraints are a front end for a list of manifolds

The constraint list is long, and the way it is presented is the important part. The documentation says these constraints are a front end for the families of spaces listed below.

The constraints are symmetric matrices, skew-symmetric matrices, vectors of norm 1, matrices with orthogonal columns, skew-symmetric matrices under the grassmannian name, matrices with singular values in the interval [1-lambda, 1+lambda], invertible matrices with positive determinant, matrices of determinant equal to 1, matrices of rank at most r, matrices of rank r, positive definite matrices, positive semidefinite matrices, and the positive semidefinite variants with rank at most r and with rank exactly r.

The spaces behind them are the more precise statement. The list covers the reals R^n for unrestricted optimization, the vector space of symmetric matrices, the vector space of skew-symmetric matrices, the sphere, the manifold of n by n orthogonal matrices, the manifold of n by k matrices with orthonormal columns, the almost-orthogonal manifold, the manifold of k-dimensional subspaces in R^n, the manifold of invertible matrices with positive determinant, the manifold of matrices with determinant equal to 1, the variety of matrices of rank r or less, the manifold of matrices of rank r, and the three cones of positive definite, positive semidefinite and positive semidefinite of bounded rank, plus the product of manifolds.

The distinction between a vector space and a manifold matters. A symmetric matrix is a linear subspace, so projecting onto it is a linear operation. A matrix of positive semidefinite type lives on a cone, and the geometry there is different, and a fixed-rank matrix is not even a manifold everywhere because the rank-deficient set is inside it.

So the constraint you pick and the space it corresponds to are not the same choice. The documentation notes that each constraint has some extra parameters which can be used to tailor the behaviour of each constraint to the problem at hand, and that the manifold implementations give the user more flexibility on the options they choose for each parametrisation.

Riemannian gradient descent is one option among many

Each constraint in GeoTorch is implemented as a manifold, and the documentation is explicit that these support Riemannian Gradient Descent but also support optimization via any other PyTorch optimizer.

That is the library's central design choice stated as a fact, and it cuts both ways.

The benefit is that a constrained layer drops into an existing training script. You do not rewrite your optimiser, your scheduler or your checkpointing. You write the model, you attach the constraint, and you train it as you always did.

The cost is that Riemannian gradient descent is not what is happening under the hood. Riemannian gradient descent is the mathematically correct way to optimise on a manifold: you take a gradient in the ambient space, project it onto the tangent space of the manifold at the current point, and take a step along it. What a reparametrisation approach does instead is parameterise the manifold, so that unconstrained optimisation in the parameter space corresponds to a constrained point in the original space.

The two are not the same algorithm, and the difference shows up in convergence behaviour and in how the constraint is exactly satisfied. With a reparametrisation the constraint is satisfied by construction at every step. With Riemannian gradient descent the iterate is on the manifold but the step direction and the learning rate interact with the curvature of the manifold, which is what makes the choice of metric matter.

The library's position is that the manifold formulation is the flexible one, because the same manifold can serve as the base for different parametrisation schemes, and the user chooses. Each space of dimension (n, k) can be applied to tensors of shape (*, n, k), which is what gives efficient parallel implementations of product spaces, with the ObliqueManifold(n,k) example given as a matrix with unit length columns, a product of spheres.

If you need the specific convergence guarantees of Riemannian gradient descent rather than a parametrisation that respects the constraint, that distinction is the thing to check against your own requirements.

The example scripts are where the actual usage is

The documentation points at the example files rather than trying to summarise them, and the examples directory is small and revealing.

There is copying_problem.py, eigenvalue.py, sequential_mnist.py, a parametrisations notebook and an __init__.py. The names alone tell you a lot about how the library is meant to be used.

A copying problem is the classic constrained optimisation example, where you want a matrix that is close to a target matrix while satisfying a structural constraint, and it is the right shape for the manifold tools because the objective is unconstrained and the constraint is on the parameter. The eigenvalue example works with spectral structure, which is where symmetric, positive definite and the fixed-rank spaces become useful, since eigenvalue problems live naturally on those sets. Sequential MNIST is a deep learning task with a matrix constraint, and it is the case where the claim about no boilerplate in the training code gets tested, because the training loop is a normal loop and the constraint is on the layer.

The parametrisations notebook is the one to look at first if you are evaluating the library, since the choice of parametrisation is the decision the library leaves to you and the notebook is where those options are laid out. A notebook is also the right format for that, because the point is to compare the schemes and the differences show up in the numbers rather than in the code.

The repository structure is a normal package layout. There is a geotorch/ directory with the implementation, a test/ directory, an examples/ directory, a docs/ directory, a pyproject.toml and a setup.cfg, plus a .readthedocs.yml for the documentation build and a MANIFEST.in.

Version 0.4 jumped the PyTorch version and sat still for years

The release history is short and the gap is the story.

Version 0.4.0 is titled Update to PyTorch 2. General improvements and was published on 2026-06-06. Before that, 0.3.0 from 2021-11-14 and 0.2.0 from 2021-06-01. So the library sat at 0.3.0 for about four and a half years and then released a 0.4.0 that moved to PyTorch 2 and did general improvements.

The project file shows the current baseline. The dependency is `torch>=2.6`, Python 3.10 or newer, and the classifiers list Python 3.10 through 3.14. The version is dynamic, taken from the `geotorch.__version__` attribute, which is the older setuptools style rather than a version in the file.

The classifier says Development Status 3 - Alpha, and with a 0.x version number that is consistent rather than an oversight. The licence is MIT, with the author listed as Mario Lezcano Casado, and the documentation is at geotorch.readthedocs.io.

For an adopter, the practical reading is that this is a research library with a small maintenance footprint that has been modernised but not churned. That is a good sign for API stability, since a constraint library has to keep working with every PyTorch release it claims to support, and a library that only updates when the framework forces it is likely to have fewer breaking changes than a busier one. It is also a sign that if you need a constraint that is not in the list, you are likely waiting for a release rather than for a pull request.

Editorial conclusion

Adopt GeoTorch if you need a layer whose weights are constrained by construction, such as an orthogonal projection, a low-rank convolution or a positive definite matrix, and you want to keep using standard PyTorch optimizers. Skip it if you need exact constrained optimisation, since the library implements it as a reparametrisation on a manifold rather than as a projected gradient step, and if you are not already on PyTorch 2.6 or newer. Verify first which space your constraint corresponds to, because the constraints are a front end for the manifold list and picking the wrong one changes the geometry rather than just the constraint, and that the documented manifolds support Riemannian gradient descent as well as ordinary optimizers.

Frequently asked questions

Official sources

1. lezcano/geotorch on GitHub
2. License: MIT
3. Project website
4. README
5. Releases