Optuna: define-by-run hyperparameter search in Python

The problem Optuna addresses, and the shape of its API

Tuning a model by hand means writing loops over candidate values, running each combination, and keeping the results somewhere you can compare them. Optuna replaces that with two named concepts. A study is the optimization run; a trial is one execution of the objective function. The README states the goal plainly: a study finds the optimal set of hyperparameter values through multiple trials, and the framework exists to automate and accelerate that process. The intended audience is machine learning practitioners working in Python, which is why the sample code imports scikit-learn and tunes a regressor choice between SVR and RandomForest. Nothing in the material suggests the library is limited to scikit-learn. The objective function is any Python callable that returns a number, so the same structure applies to gradient boosting libraries, neural network training scripts, or simulation code that happens to be callable from Python. The unit of adoption is small: one function, one create_study call, one optimize call.

Define-by-run: search spaces as control flow, not as declarations

The distinguishing design choice is the imperative, define-by-run API. The README describes it as letting the user dynamically construct search spaces. In the sample, the trial object is asked for a categorical value first, and that value determines which subsequent parameters are requested: if the regressor is SVR, the code suggests svr_c on a log scale between 1e-10 and 1e10; otherwise it suggests rf_max_depth as an integer between 2 and 32. This is conditional search space construction, and it is the reason the API is Pythonic rather than a configuration schema. A declarative format would need a way to express that rf_max_depth is meaningless when the regressor is SVR. Optuna sidesteps the problem by making the space a side effect of running the function. That is a genuine advantage for irregular spaces, and it is also the source of the framework's main constraint: the search space exists only while the objective runs, so anything that inspects or reproduces the space has to execute the objective or replay a recorded trial.

The trial loop and what the sampler actually sees

Each call to a suggest method on the trial object both proposes a value and records it. The objective returns a scalar, and the README notes that this value is linked with the trial. That link is the entire feedback channel. The sampler receives a history of parameter sets and their resulting objective values, and uses that history to propose the next set. The README points to state-of-the-art algorithms for sampling hyperparameters and for pruning unpromising trials, and links a tutorial page on efficient optimization algorithms, but it does not name the default sampler or describe its internal model. If you need to know which algorithm runs by default, that detail lives in the documentation rather than the README, and you should read it before assuming any particular behavior. Pruning is the second half of the loop: the README lists pruning as a separate capability from sampling, and the examples repository is said to cover pruning setups. Pruning requires the objective to report intermediate values, which the README's minimal example does not do. A reader who copies only the sample code gets sampling without pruning.

Installation and the smallest working setup

Installation is one command. The README gives pip install optuna for PyPI and conda install -c conda-forge optuna for Anaconda Cloud. Python 3.9 or newer is required, and the badge list names 3.9 through 3.14. The README claims a simple installation with few requirements, which is consistent with a library whose core loop is pure Python. The minimal program is four lines of setup around the objective: optuna.create_study() with no arguments, then study.optimize(objective, n_trials=100). No configuration file, no database, no server. That default path is in-memory and single-process. The README's distributed tutorial is where storage and worker coordination appear, and the README explicitly frames parallelization as scaling studies to tens or hundreds of workers with little or no code change. The phrase little or no code change is doing real work there: the objective stays the same, but the study creation call does not, because workers need shared state. Treat the single-process example as the tutorial and the distributed setup as the deployment.

Where Optuna is the wrong tool

The README's own framing sets the boundary. This is a framework for automation and acceleration of optimization studies, and the sample objective is cheap: fetch a dataset, fit a small model, compute mean squared error. That shape suits Optuna because the sampler needs many trials to build a useful model of the space. If a single objective evaluation takes hours, the number of trials you can afford collapses, and a sampler that learns from history has little history to learn from. The README does not discuss this regime. A second limitation follows from define-by-run: because the space is constructed at runtime, there is no static artifact describing it. Teams that need to review a search space before execution, or share it with a scheduler written in another language, get no file to hand over. The README also does not describe what happens when two workers request the same trial, or how a crashed worker's trial is reclaimed. Those are storage-backend questions, and the README defers them to the distributed tutorial. Finally, the README lists pruning as a feature but the sample code never calls a pruning API, so the feature is invisible to anyone who stops at the quickstart.

Rustuna and the cost of Python-level sampling

The v5.0.0 announcement in the README pairs Optuna v5 with Rustuna v0.1.0, described as a faster implementation of Optuna written in Rust that keeps the API you already know and rewrites the parts that start to hurt at scale: sampling speed and memory efficiency. The README claims Rustuna supports TPE, MOTPE, NSGA-II, and CMA-ES and finishes the same study several times to several hundred times faster for cheap objective functions. That qualifier matters. The speedup is framed for cheap objective functions, which is exactly the case where sampler overhead is a visible fraction of total runtime. If your objective is expensive, the sampler is not your bottleneck and the rewrite buys you little. The README also says Rustuna's storage is designed to be memory-efficient and that discarding unnecessary trial history prevents memory growth, but the sentence is truncated in the supplied material, so the mechanism is not fully described here. What the README does not state is whether Rustuna is a drop-in replacement, a separate package, or a future default. The announcement links a separate repository, which suggests a parallel project rather than a change inside Optuna. Verify that before planning a migration.

Alternatives and the difference in approach

The clearest contrast is with declarative search-space tools such as scikit-learn's own search utilities, where you pass a dictionary of parameter grids or distributions and the search space is data rather than code. That approach cannot express the conditional in the README sample, where rf_max_depth only exists on one branch. It can express it only by enumerating full configurations and accepting that some parameters are ignored. Optuna's define-by-run model handles the branch naturally and pays for it by making the space non-inspectable without execution. A second alternative is to treat the objective as a black box and use a general-purpose optimizer outside Python, which decouples the search from the model code but requires serializing parameters across a process boundary and gives up the trial object's intermediate-value reporting that pruning depends on. Optuna's position is that the search lives inside the language that defines the model. That is convenient and it is also a lock-in: the objective, the search space, and the framework share one runtime.

Maintenance, versioning, and the MIT licence

Optuna ships under the MIT licence, which the README displays as a badge and which permits commercial use and modification with the licence text retained. That is a permissive arrangement and it removes licence negotiation from the adoption path. This is not legal advice; read the licence text yourself if the distinction matters to your organization. On maintenance, the release cadence visible in the README is roughly every six to eight weeks for minor versions through 2025 and into 2026, with v4.9.0 in June 2026, a v5.0.0 release candidate in August 2026, and v5.0.0 in September 2026. The repository is not archived and the last push timestamp is 2026-09-10, three days after the v5.0.0 release. A major version bump plus a companion Rust project means upgrade cost is not zero: the README does not state what breaks between v4 and v5, so the release notes are the place to check before upgrading a production study. The practical upgrade cost is the cost of re-reading the release notes and re-running a study, not a rewrite.

Editorial conclusion

Adopt Optuna if your search space is naturally expressed as Python branching and your objective is a single scalar you can compute repeatedly. Do not adopt it if you need a declarative configuration format shared with non-Python tooling, or if each objective evaluation costs hours and you have no pruning signal to cut trials early. Before committing, verify two things on your own workload: that your storage backend survives a worker crash mid-trial, and that your objective returns a value cheap enough that the sampler's own overhead stays in the noise. Start with the pip install optuna path and a single-process study, then move to a shared RDBStorage URL only once you have measured the per-trial cost.

Official sources

1. License: MIT
2. optuna/optuna on GitHub
3. Project website
4. README
5. Releases