# BOAT: composing bi-level optimization solvers from 19 gradient primitives

> BOAT is a PyTorch framework for gradient-based bi-level optimization that splits solvers into gradient mapping, numerical approximation and first-order stages, so a solver variant is a configuration choice rather than a new implementation.

**callous-youth/BOAT** — A Compositional Operation Toolbox for Gradient-based Bi-Level Optimization

- Repository: https://github.com/callous-youth/BOAT
- Website: https://boat.readthedocs.io/en/latest/
- Stars: 1,142 · Forks: 159
- Language: Python
- License: MIT
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/callous-youth-boat

## Why one solver routine is not enough for bi-level problems

Bi-level optimization is the setting where one optimization problem is nested inside another, and the outer problem is solved through the inner one. Hyper-parameter tuning, meta-learning and neural architecture search all fall into it. The difficulty is that the inner problem has no closed-form solution, so the outer gradient has to be estimated somehow, and the literature offers many ways to estimate it.

BOAT is a response to a specific complaint about existing libraries: they typically encapsulate fixed solver routines, so trying a different estimation method means writing a new solver rather than swapping a component. The project's framing is compositional and operation-level, aimed at closing the gap between theoretical modelling and practical implementation.

The stated result of that design is that 19 atomic gradient operations can be dynamically serialized into more than 85 solver variants, spanning what the project calls the full spectrum of BLO methodologies. The claim to check when adopting is not the count but the mechanism: everything is expressed through a unified constraint reconstruction perspective, so the operations are meant to be interoperable rather than merely co-located.

The primary implementation is PyTorch based. Two other backends exist on separate branches, a Jittor version described as an accelerated version with meta-operators, and a MindSpore version aimed at ascending chips and industry scenarios. The PyTorch package is published as boat-torch and the other two as BOAT_Jittor and BOAT_Mindspore, with all three at version 1.1.1 released on 2026-08-15.

## Three stages, and where the difficulty actually lives

The abstraction is a decomposition into three modular stages, and each stage corresponds to a distinct place where a BLO solver can go wrong.

Gradient Mapping, the GM-OL library, reconstructs the lower-level iterative trajectory. It customises the dynamic mapping rules that shape the optimization path and the variable coupling, and it contains four atomic operations: NGD, naive gradient descent; GDA, gradient descent aggregation; DI, dynamic initialization; and DM, the dual multiplier or KKT route.

Numerical Approximation, NA-OL, resolves what the project calls the auxiliary gradient bottleneck, approximating the implicit gradients or hyper-gradients. It offers the widest set of primitives, with RAD for reverse-AD or unrolled differentiation, RGT for reverse gradient truncation, PTT for pessimistic trajectory truncation, FD for finite difference as used in DARTS, CG for conjugate gradient, NS for the Neumann series, IGA for implicit gradient approximation, IAD for init-based AD as used in MAML, and FOA for first-order approximation as used in Reptile.

First-Order, FO-OL, constructs single-level surrogates, reformulating the nested problem into first-order objectives through value functions or penalties so that Hessian computations are avoided. Its operations are VSO, VFO, MESO for the Moreau envelope, PGDO, ALTO for alternating optimization, and GAFFO.

The split is not cosmetic. If you are chasing a slow inner loop, the operation to change is in NA-OL. If your outer problem is not moving, the mapping in GM-OL is the place to look. A library that hides this structure behind a single solver name forces you to read the algorithm to find the same information.

## Solver variants as JSON configuration instead of code

Because the stages are separate, a solver variant is a matter of choosing operations. The project describes this as generative solver construction, with dynamic serialization of operations, and says that you can recover classical algorithms such as DARTS and MAML, or build hybrid solvers that do not have a paper attached, by changing configurations.

The mechanism is configuration driven. Complex optimization strategies are defined through JSON configurations, with a boat_config and a loss_config, which decouples the algorithmic logic from the model definitions. The repository carries a configs directory at the top level, and the example directories are named after the problem families they cover: L2_Reg, data_hyper_cleaning, meta_learning and neural_architecture_search.

That separation has an obvious upside for research code, where a solver comparison means running the same model under several configurations and tabulating results rather than editing a training loop each time. It also has a cost, which is worth being clear about. When a run fails, the traceback points at framework internals rather than at the algorithm, and the person reading the result has to read the JSON to know what was actually run. A configuration that describes an algorithm is only easier to reason about than the code it replaces if the configuration is also readable.

The framework also offers a unified computational analysis, a complexity analysis of gradient-based BLO techniques intended to guide configuration choice for efficiency and accuracy. The project describes the testing setup as pytest based, with a stated 99 percent code coverage figure and a codecov integration in the repository, plus a GitHub Actions workflow.

## Setting up the environment and the PyTorch dependency floor

The installation path is a Python environment plus the package, and the project recommends a virtual environment. The documented conda commands are:

```bash
conda create -n boat python=3.12
conda activate boat
```

The package itself is distributed on PyPI under the name boat-torch, which matches the name in the packaging metadata, and the version there is 1.1.1.

The dependency floor is where the friction lives. The requirements file pins numpy at 1.26.4 or lower, with the comment that it is the fundamental library for numerical computations, and requires torch at 1.4 or newer as the primary deep learning framework. It also pulls in higher, the library for higher-order optimization in PyTorch, which is the piece that makes the NA-OL operations that need second-order information possible at all, plus matplotlib and torchvision.

The numpy ceiling is the detail to plan around. A project that has to run on numpy 2.x cannot take this dependency set as it stands, and the constraint arrives through pip resolution rather than through a clear error, which is the more awkward way to discover it. The packaging metadata declares python_requires of 3.8.0 or newer and lists classifiers for Python 3.8 through 3.12, so the conda recipe using 3.12 sits at the top of the supported range.

Some examples need extra packages. The L2_Reg example additionally requires scikit-learn, installed with pip install scikit-learn, and the example directories are listed in the repository, so check the requirements of the specific example before running it rather than after.

## What the repository contains beyond the library

The top level of the repository reads like a research project with a library bolted on, and that is a fair description. Alongside the boat_torch package there are docs and a _static directory, the examples directory, the configs directory, a requirements.txt, a setup.py, a Makefile and a make.bat, a MANIFEST.in, a CHANGELOG.md, a CODE_OF_CONDUCT.md, a CONTRIBUTING.md, a SECURITY.md and a LICENSE.

The Makefile is not a build system for the package, it is a thin Sphinx wrapper. It sets SPHINXOPTS and SPHINXBUILD to sphinx-build, points SOURCEDIR at docs/source and BUILDDIR at docs/build, and the default help target prints the sphinx-build help. Every other target is routed through a catch-all rule that invokes sphinx-build with the requested target name, which is the standard trick for getting sphinx-build's built-in targets like html or linkcheck through make. There is also a .readthedocs.yaml, so the documentation site is built by Read the Docs, and the project homepage points at boat.readthedocs.io.

So the documentation, not the tests, is what make serves. The tests are run through pytest with a codecov configuration, and the release process is separate per backend, with a PyTorch release tagged v1.1.1, a MindSpore release tagged boat-ms-v1.1.1 and a Jittor release tagged boat-jit-v1.1.1, all on 2026-08-15.

For an adopter this matters in one specific way: the three backends are versioned independently on independent branches, so the Jittor and MindSpore trees are not the same codebase with a different import name. A capability described in the PyTorch documentation should not be assumed to exist in the others.

## Where a compositional research framework is the wrong tool

The first limitation is that a compositional abstraction optimises for exploring, not for shipping. If you need one solver, well understood and tuned, the configuration layer is overhead: a JSON file and a framework between you and the update rule, with the result that reading a training curve no longer tells you what the algorithm did. For a production training job, a fixed solver in a script you own is easier to reason about.

The second limitation is the learning cost of the abstraction itself. Understanding why a particular composition is right for your problem requires understanding the three stages and the operation catalogue, which is a substantial amount of reading before you get a result. The project does help here with the complexity analysis and the per-operation papers, but neither removes the need to read.

The third is the dependency and platform situation. Three backends maintained on separate branches means three maintenance surfaces, and the PyTorch version pins numpy at 1.26.4 or lower, which dates the numerical stack. The Jittor branch is described as accelerated with meta-operators, and the MindSpore branch is aimed at ascending chips, which is a signal about the intended deployment environment rather than about general availability.

On maintenance, the repository is not archived and the last push was on 2026-08-15, with all three backends released at 1.1.1 on the same day. The project is MIT licensed, with the licence recorded both in the packaging metadata and in a LICENSE file at the top level.

## Conclusion

BOAT fits a research group working on bi-level optimization problems such as hyper-parameter tuning, meta-learning or neural architecture search, where the interesting question is which algorithmic combination to try rather than which loop to write. It is a poor fit for a production training job that just needs one well-understood solver, because the abstraction adds a configuration layer between you and the algorithm. Before adopting, read the complexity analysis the project provides to understand what a given composition costs, run one of the example directories end to end to see the configuration format, and check that your PyTorch and Python versions are covered by the declared support, which is Python 3.8 and newer with numpy pinned at 1.26.4 or lower.

## FAQ

### What is BOAT used for?

BOAT is a compositional, operation-level framework for gradient-based bi-level optimization, designed to bridge the gap between theoretical modelling and practical implementation. It factors the bi-level optimization workflow into atomic reusable primitives rather than fixed solver routines.

### How do I install BOAT?

Create a virtual environment, for example with conda create -n boat python=3.12 followed by conda activate boat, then install the PyTorch package boat-torch from PyPI. The package requires torch 1.4 or newer and pins numpy at 1.26.4 or lower, and some examples need extra packages such as scikit-learn.

### What are the three stages of a BOAT solver?

Gradient Mapping reconstructs the lower-level iterative trajectory, Numerical Approximation resolves the implicit or hyper-gradient estimation bottleneck, and First-Order constructs single-level surrogates that avoid Hessian computations. Each stage has its own library of atomic operations.

### How do I configure a solver in BOAT?

Solvers are defined through JSON configurations, with a boat_config and a loss_config, which separates the algorithmic logic from the model definitions. The repository ships a configs directory and example directories for L2_Reg, data_hyper_cleaning, meta_learning and neural_architecture_search.

### Which backends does BOAT support?

The main branch is the PyTorch version published as boat-torch. A Jittor based version with meta-operators and a MindSpore based version aimed at ascending chips live on separate branches, published as BOAT_Jittor and BOAT_Mindspore, and each is versioned and released independently.

## Sources

- [callous-youth/BOAT on GitHub](https://github.com/callous-youth/BOAT)
- [License: MIT](https://github.com/callous-youth/BOAT/blob/main/LICENSE)
- [Project website](https://boat.readthedocs.io/en/latest/)
- [README](https://github.com/callous-youth/BOAT/blob/main/README.md)
- [Releases](https://github.com/callous-youth/BOAT/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/callous-youth-boat