EvoTorch: evolutionary search where the objective is just a PyTorch tensor function

One abstraction for problems with and without gradients

EvoTorch is an evolutionary computation library built directly on PyTorch, developed at NNAISENSE, and its pitch is that the same machinery solves black-box optimisation, reinforcement learning and supervised learning whether or not the problem is differentiable.

The unit of work is a Problem object. You give it an objective function, and the library handles the population, the mutation operators, the evaluation loop and the bookkeeping. The README's claim about not requiring differentiability is the substantive one: gradient-free search is the only option for a simulator, a black-box API, or a controller with discrete actions, and this is a library built for exactly that case rather than a fallback mode.

Because everything is expressed in PyTorch operations, a population is a tensor and an evaluation is a batched call, which is what makes GPU execution and distributed execution possible without a separate worker system. The library also offers Ray for scaling, splitting work across multiple CPUs, multiple GPUs, or multiple machines in a Ray cluster.

It is a focused library: algorithms, problems, neuroevolution helpers and loggers. There is no experiment manager, no model zoo and no pipeline abstraction, which is a reasonable scope for something intended to sit inside a research codebase you already have.

Vectorized or not: the one decision that shapes your code

There are two ways to write an objective, and the distinction explains most of the API.

A non-vectorized objective receives a single solution as a 1-dimensional torch tensor and returns a fitness as a scalar. A vectorized objective receives a batch of solutions as a 2-dimensional tensor and returns a 1-dimensional tensor of fitnesses. The Problem is constructed with vectorized=True or False, and your function has to match.

The canonical example is Rastrigin, where the sum is taken along dimension 1 so that a batch of rows collapses to one fitness per row:

def rastrigin(x: torch.Tensor) -> torch.Tensor:
    A = 10
    (_, n) = x.shape
    return A * n + torch.sum((x**2) - A * torch.cos(2 * math.pi * x), 1)

That trailing 1 is the whole vectorisation contract. Write the sum over the wrong axis and the function still runs, still returns a tensor, and quietly returns one number for the batch, which is a bug that shows up as a search that never converges rather than as an exception.

The Problem constructor is also where the problem definition lives, including the search bounds and the solution length:

problem = Problem(
    "min",
    rastrigin,
    initial_bounds=(-5.12, 5.12),
    solution_length=100,
    vectorized=True,
    # device="cuda:0"  # enable this line if you wish to use GPU
)

The optimisation direction is the first argument, a string, which is what lets the same algorithms handle minimisation and maximisation.

Two algorithm families, and what the split tells you

The catalogue divides cleanly, and knowing which family you are in predicts how you will use it.

Distribution-based algorithms keep a search distribution, usually a mean vector plus a scale, and update it from fitness statistics rather than from an explicit population of individuals. The list is PGPE for Policy Gradients with Parameter-based Exploration, XNES, CMA-ES, SNES, and CEM, the Cross Entropy Method. These are the ones that fit gradient-free continuous optimisation, and CMA-ES in particular is the reference point most people compare against.

Population-based algorithms keep individuals. The GeneticAlgorithm implementation also supports multiple objectives, in which case it behaves like NSGA-II, which makes it usable for Pareto fronts without a second library. CoSyNE covers cooperative synapse neuroevolution, where fitness is assessed by combining sub-networks, and MAPElites maintains a multi-dimensional archive of phenotypic elites, which is how you get diversity rather than a single optimum.

The distinction has a practical consequence beyond theory. A distribution-based search has a centre you can extract at the end, which is what makes it suitable for evolving a policy, and PicklingLogger exists partly so that centre is saved. A population-based method like MAP-Elites gives you an archive of interesting solutions instead, and asking it for one best point is the wrong question.

Swapping algorithms is a constructor change on the same Problem, which is the cleanest sign that the abstraction holds up.

A complete black-box run, with logging attached

The README example wires the whole loop together: problem, algorithm, two loggers, a run and a plot.

from evotorch.algorithms import SNES
from evotorch.logging import StdOutLogger, PandasLogger

searcher = SNES(problem, popsize=1000, stdev_init=10.0)

_ = StdOutLogger(searcher, interval=10)
pandas_logger = PandasLogger(searcher)

searcher.run(2000)

pandas_frame = pandas_logger.to_dataframe()
pandas_frame["best_eval"].plot()

Two loggers do different jobs. StdOutLogger prints progress every ten generations, which is what you want in a terminal, and PandasLogger accumulates history in memory for later analysis, exposed through to_dataframe. Since the second one grows with the number of generations, a run of a few thousand generations is cheap while a long one becomes a memory decision.

The algorithm constructor takes the population size and the initial spread, stdev_init=10.0 here, and run takes a generation count rather than a time or a fitness target. That is a coarse stopping criterion, which is fine for a benchmark like Rastrigin and awkward for a real workload where you would rather stop on improvement.

Switching to a GPU is one commented line in the Problem, device="cuda:0", rather than a separate code path.

GymNE: evolving a policy instead of training one

The reinforcement learning entry point is GymNE, and it removes most of the boilerplate of neuroevolution by building the network and the environment wrapper for you:

problem = GymNE(
    env="Humanoid-v4",
    network="Linear(obs_length, act_length)",
    observation_normalization=True,
    decrease_rewards_by=5.0,
    num_actors="max",
)

Each argument is a decision. The network is given as a string, so obs_length and act_length are placeholders the problem fills in from the environment. observation_normalization handles the scale difference that otherwise dominates linear policies. decrease_rewards_by=5.0 shifts reward, which matters for algorithms that treat the magnitude as meaningful. num_actors="max" uses every available CPU, with a comment noting you can set 4 instead.

The algorithm then carries the population mechanics, and the comments in the example are worth reading because they document the trade-offs:

searcher = PGPE(
    problem,
    popsize=200,
    num_interactions=int(200 * 1000 * 0.75),
    popsize_max=3200,
    center_learning_rate=0.0075,
    stdev_learning_rate=0.1,
    radius_init=0.27,
    optimizer="clipup",
    optimizer_config={"max_speed": 0.15},
)

num_interactions is a simulator budget rather than a generation budget: keep sampling until the threshold is met, stop growing the population once popsize_max is reached. That is how you bound the cost of a run that would otherwise scale with popsize times generations. The ClipUp optimizer with a max_speed cap is the other notable piece, bounding how far the distribution centre can move per step, which is the usual defence against a search that diverges mid-run.

PicklingLogger is attached here too, and the README notes that among the saved results is the centre of the search distribution, since PGPE is distribution-based.

Scaling: Ray, GPUs and the shape of the project

Because evaluation is a batched tensor operation, GPU execution is a device argument rather than a rewrite. Distribution across machines comes from Ray, and the README lists the three axes explicitly: multiple CPUs, multiple GPUs, and multiple computers in a Ray cluster.

That distinction is worth holding onto. Vectorisation inside one process is a tensor shape; distribution is a scheduling problem delegated to Ray. EvoTorch's contribution is in the first, and it hands the second to a framework that already solves actor placement, fault tolerance and result collection.

The repository is a conventional modern Python package. pyproject.toml configures pytest with coverage enabled against the evotorch package, sets a line length of 120 for both black and ruff, and configures isort with the black profile. Version numbers come from setuptools_scm, which writes src/evotorch/__version__.py at build time with a fallback of 0.0 when git metadata is unavailable. There is a noxfile.py for session automation, an env.yml for a conda environment, and mkdocs.yml with the documentation built and tested in separate workflows.

Terms are Apache 2.0, stated in the LICENSE file and repeated in the setup.py header.

Where EvoTorch is the wrong tool

The honest weakness is packaging cadence, not the design.

Packaging is irregular. Versions 0.6.0 and 0.6.1 were both published on 2025-05-14, hours apart, the previous one before them is 0.5.1 from 2023-11-02, and the repository itself was pushed to on 2026-08-31. That gap is the practical consequence: pip install evotorch gives you May 2025, while fixes and features since then exist only on master. If you install from git you take on the unreleased state, and if you pin an installed version you may be missing fixes you would want.

The second boundary is environment coupling in GymNE. The example names Humanoid-v4, and gym environment identifiers are versioned strings that get renamed when gym changes its environment set. A neuroevolution problem wrapping an env that no longer resolves under your installed gym is a dependency problem, not an algorithm problem, and it will surface as a construction failure.

The third is scale of effort. These searches are slower per unit of compute than gradient methods when gradients exist, and the library is not trying to change that. If your problem is differentiable and you can backpropagate, evolutionary search is a research tool, not the fastest route to a result. Use it where the objective is opaque.

The fourth is that vectorisation is optional rather than mandatory, which is friendly, and also means the fast path is easy to miss. A non-vectorized run on one CPU is a correct EvoTorch program and an unusably slow one.

Editorial conclusion

Adopt EvoTorch when your objective resists gradients, when you are evolving a network rather than training one with backpropagation, or when you want CMA-ES or MAP-Elites to run on the GPU without reimplementing the batch dimension yourself. The Problem abstraction is the reason it feels cheap to try: a non-vectorized objective takes one 1-D tensor and returns a scalar, a vectorized one takes a 2-D batch and returns 1-D fitnesses, and the same function object serves both. Do not adopt it expecting a mature ecosystem around a fast release cadence, because the packaged versions are sparse and the newest tag is well behind the working tree, which means pinning an old release may cost you fixes that only exist on master. Verify four things first. Install with pip install evotorch and confirm the version you get, since pyproject.toml derives __version__ from setuptools_scm with a fallback of 0.0 for trees without tags. Decide vectorized=True or False deliberately, because it changes how your objective must be written and a mismatch shows up as a shape error rather than a wrong answer. Check your gym environment version against what GymNE expects, since the README example pins Humanoid-v4 and gym environment identifiers move. And decide your logging before a long run, because PicklingLogger is what saves the distribution centre for a distribution-based algorithm and a run you did not log is a run you cannot resume. Start with the Rastrigin example and SNES at popsize 1000.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. nnaisense/evotorch on GitHub
3. Project website
4. README
5. Releases