Optuna: a define-by-run hyperparameter optimizer for Python

The problem Optuna solves: search spaces that only exist at runtime

Grid search needs a fixed set of axes before the run starts. Real model code rarely looks like that. A scikit-learn pipeline may pick SVR or RandomForest, and only the chosen branch has hyperparameters worth tuning. Writing that as a static grid means enumerating combinations that the code will never execute.

Optuna's answer is the define-by-run API. Inside the objective function you call trial.suggest_categorical, trial.suggest_float, trial.suggest_int, and the search space is whatever those calls produce on that trial. The README's own example branches on regressor_name and only then suggests svr_c or rf_max_depth. Conditionals and loops are ordinary Python, so the space can depend on data, on an earlier suggestion, or on a config file.

The intended audience is a Python user with a training script and a metric to minimize or maximize. The README describes the framework as particularly designed for machine learning, though nothing in the API requires a model: any function returning a number works. The vocabulary is study for one optimization run and trial for a single execution of the objective.

How a study, a trial and a sampler fit together

optuna.create_study() builds a study object. study.optimize(objective, n_trials=100) calls the objective repeatedly, once per trial, and each call receives a Trial that carries the suggested values and the returned objective value. The study keeps the history.

The sampler decides which values to try next. The README points to a page on efficient optimization algorithms and the related searches show TPE is the sampler people look for most; Optuna ships several, and custom samplers can be plugged in. Pruning is the second lever: an objective can report intermediate values and let the study stop unpromising trials early, which matters when a single trial trains a model for minutes.

Parallelism follows from the storage layer. The README lists easy parallelization among the key features and says studies scale to tens or hundreds of workers with little or no code change. The dependency list includes SQLAlchemy and Alembic, which is consistent with a relational database acting as the shared store between workers. The README does not spell out the storage URLs in the excerpt available here, so check the distributed tutorial before designing a multi-machine setup.

Visualization is part of the package: the README advertises inspecting optimization histories through a set of plotting functions, and the project also ships an optuna-dashboard package that is not described in the README text shown.

Install Optuna and run a first study

The README gives two install paths, PyPI and conda-forge, and states that Optuna supports Python 3.9 or newer. Pick one.

$ pip install optuna

$ conda install -c conda-forge optuna

After installation, the smallest useful program is the README's scikit-learn example. It defines an objective, suggests a categorical choice, branches into two model types, and optimizes for 100 trials. The value returned by the objective is what the study minimizes by default.

import optuna
import sklearn

def objective(trial):
    regressor_name = trial.suggest_categorical("regressor", ["SVR", "RandomForest"])
    if regressor_name == "SVR":
        svr_c = trial.suggest_float("svr_c", 1e-10, 1e10, log=True)
        regressor_obj = sklearn.svm.SVR(C=svr_c)
    else:
        rf_max_depth = trial.suggest_int("rf_max_depth", 2, 32)
        regressor_obj = sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)
    return 0.0

study = optuna.create_study()
study.optimize(objective, n_trials=100)

The snippet above keeps the structure of the README example but returns a constant, so it runs without a dataset. In a real run the objective trains the model, computes an error such as mean_squared_error on a validation split, and returns that number. Expect a progress bar during optimization and a study object afterwards holding every trial; the README points to plotting functions for inspecting that history, and to the optuna/optuna-examples repository for multi-objective, constrained, pruning and distributed variants.

Where Optuna costs you: cheap objectives, non-Python stacks and storage

The TPE-style samplers that make Optuna useful need history to learn from. If one evaluation takes milliseconds, the sampler overhead and the database round trips can dominate, and a random search or a hand-written sweep will finish sooner. The README itself frames the Rust rewrite around this: Rustuna is described as finishing the same study several times to several hundred times faster for cheap objective functions. That sentence is also an admission that the Python implementation is not the fast path for that workload.

Language is the second boundary. Optuna is a Python package with a Python API; the objective is a Python callable. If your training code is C++, Java or Go, you are writing a bridge before you write a search space.

The third cost is operational. Parallel workers need shared storage, and the dependency list points at a SQL database rather than a file. That is a real deployment decision: a shared database is another service to run, back up and migrate, and Alembic is present precisely because the schema evolves between versions. A single-process study on the default in-memory storage avoids all of this, but it also cannot be resumed after the process exits.

Optuna versus GridSearchCV and random search

scikit-learn's GridSearchCV is the tool most people are leaving when they arrive at Optuna, and the difference is structural rather than a matter of tuning quality. GridSearchCV takes a parameter grid as a dictionary, runs every combination, and cross-validates each one. The search space is data, fixed before execution, and the cost grows multiplicatively with the number of axes.

Optuna's space is code. That lets the objective skip branches that do not apply, sample from continuous distributions on a log scale, and stop trials early. It also lets the study continue from stored history, which GridSearchCV does not do across processes. The trade is that you write and debug an objective function instead of a parameter dictionary, and you own the storage and the parallelism story.

Random search sits between the two. It handles continuous spaces and costs nothing to set up, but it never uses the results of earlier trials. Optuna's samplers do, which is the whole reason the framework exists; if your budget is a few dozen evaluations, that advantage may not pay for the extra moving parts.

Maintenance, licensing and the v5.0.0 upgrade

The repository is not archived and the last push was on 2026-09-10, days before this writing. Release cadence is visible in the README news list: 4.7.0 in January 2026, 4.8.0 in March, 4.9.0 in June, a v5.0.0 release candidate in August, and v5.0.0 on 2026-09-07. That is a project shipping on a steady schedule, not one coasting on an old tag.

The same news entry announces Rustuna v0.1.0, a Rust implementation that the README says keeps the API you already know and rewrites sampling speed and memory efficiency. Rustuna is a separate repository at version 0.1.0, so it is early; the README lists TPE, MOTPE, NSGA-II and CMA-ES as its supported samplers and describes memory-efficient storage that discards unnecessary trial history. Treat it as a parallel track to evaluate, not as a drop-in you should assume is production-ready at 0.1.0.

Licensing is MIT, per the badge and the classifier in pyproject.toml. That is permissive and compatible with commercial use, but the repository also carries a LICENSE_THIRD_PARTY file, so anyone redistributing a bundled artifact should read it rather than assume a single licence covers everything. This is not legal advice; check the file yourself.

Upgrade cost centres on the major version. Moving from 4.x to 5.0.0 crosses a major boundary, and the storage schema is versioned through Alembic, so an existing database-backed study is the thing to verify first. The README does not document rollback.

Who should adopt Optuna, and who should not

Adopt it when the objective is already a Python function, the search space has branches or continuous ranges, and a single evaluation is expensive enough that learning from prior trials beats trying combinations blindly. That covers most scikit-learn, XGBoost and PyTorch training loops, and the optuna-examples repository exists for exactly those setups.

Do not adopt it when evaluations are near-instant, when the training stack is not Python, or when you want the tuning infrastructure hosted for you. In those cases the sampler cannot earn its overhead, the API forces a bridge, or the database becomes someone else's problem.

Before you commit, verify three things in this order: that your interpreter is 3.9 or newer, that you have chosen a storage backend you are willing to operate if you plan to run more than one worker, and that v5.0.0's changes do not affect the samplers your study depends on. The release notes for v5.0.0 are the place to confirm the third.

Editorial conclusion

Adopt Optuna if your objective is a Python function and you want to move past grid search without leaving your training code. Skip it if the optimizer must live outside Python or if you need the tuning loop managed for you. Before committing, confirm the Python version floor of 3.9, decide which storage backend your workers will share, and check whether v5.0.0 changed anything your existing samplers rely on.

Frequently asked questions

Official sources

1. License: MIT
2. optuna/optuna on GitHub
3. Project website
4. README
5. Releases