DeepHyper: Bayesian HPO Built for MPI-Scale Job Queues

The problem DeepHyper targets: search that outlives one process

Most hyperparameter optimization libraries assume the objective function can be called in-process. You hand a sampler a callable, it calls the callable, it gets a number back. That model breaks the moment each evaluation is a training run that must be submitted to a scheduler, occupy a node for hours, and return a metric through a file or a log. DeepHyper's central abstraction, the Evaluator, exists to absorb that difference. The README describes the run function as a black box that takes a job object and reads its parameters from job.parameters, then states that this function is bound to an Evaluator that is responsible for distributing the computation of multiple evaluations. The search object never calls your training code directly. It asks the evaluator for candidates and receives results.

The audience follows from that design. The topics list includes hpc and mpi alongside automl and neural-architecture-search, and the acknowledgments name a U.S. Department of Energy Early Career Award and the Argonne Leadership Computing Facility. This is a library written by people who run searches on supercomputers, and the interface reflects it. If your evaluations are cheap and local, the evaluator indirection is overhead you pay for nothing. If they are expensive and remote, it is the part that makes the rest usable.

The evaluator boundary and what CBO actually returns

The quickstart shows the full data flow in about twenty lines. A problem object is built by calling HpProblem and then add_hyperparameter three times: a real parameter over (-10.0, 10.0) named x, a discrete parameter over (0, 10) named b, and a categorical parameter over the list ["linear", "cubic"] named function. The evaluator is constructed with Evaluator.create(run, method="process", method_kwargs={"num_workers": 2}). The search is CBO(problem, log_dir=tmp_path, random_state=42), and execution is search.search(evaluator, max_evals=100).

Note the direction convention. The README states in bold that the search MAXIMIZES the return value of run(job). Anyone porting an existing minimization objective has to negate it, and the quickstart's own assertion, abs(results.objective.max()) > 1000, only makes sense under maximization.

The returned object is not a list of scores. The printed frame has twelve columns, including p:b, p:function and p:x for the proposed parameters, objective and job_id and job_status per evaluation, m:timestamp_submit and m:timestamp_gather for timing, and a parallel sol.* family (sol.p:b, sol.p:function, sol.p:x, sol.objective) holding the incumbent solution at each row. That last group is the useful part for post-hoc analysis: you can reconstruct the optimization trajectory rather than only reading the final answer. The search is seeded through random_state, which the quickstart sets to 42, so a run is reproducible given the same evaluator behaviour. Reproducibility across a real scheduler is a different question, and the README does not address it.

Installing and running the quickstart as written

Installation is a single command, pip install deephyper, with the README pointing to the Installation page in the documentation for anything beyond that. The example is written as a pytest function named test_quickstart that takes tmp_path and is invoked manually through test_quickstart(".") under a __main__ guard, which is an unusual way to ship a first example. It works, but a reader who copies the body without the surrounding function loses tmp_path and the log_dir argument with it.

The config keys that matter in the snippet are few: method="process" and method_kwargs={"num_workers": 2} on the evaluator, log_dir and random_state on CBO, and max_evals on search. The README does not enumerate the other evaluator methods or their keyword arguments, so the process evaluator shown here is the only one I can describe from the supplied material. The output block shows job_status values of DONE and timestamps that climb from roughly 0.01 seconds to roughly 36 seconds across 101 rows, which is consistent with two workers processing a queue, though the README presents it as sample output rather than as a benchmark.

The final block reports the optimum as function cubic, x approximately 9.99958, b equal to 10, and y approximately 1009.87. That is the expected answer for a monotone cubic over the stated bounds, which makes the example a correctness check rather than a demonstration of search efficiency.

Where the abstraction costs you: local work and undocumented evaluators

The honest limitation is that the README sells breadth it does not document. The description promises neural architecture search, multi-fidelity and ensemble capabilities, and the topics list adds uncertainty-quantification and raylib, but the README body contains no example of any of them. A reader arriving from the repository page has one worked example, the CBO search on a three-parameter synthetic function, and a link to the documentation site. Whether multi-fidelity scheduling is configured through a search argument, an evaluator argument, or a separate class cannot be determined from the material provided.

The second limitation is structural. Every evaluation passes through the evaluator boundary, and in the process method that means serialization of the job and its result across process boundaries. For a synthetic function returning a float this is invisible. For an objective that returns a large artifact, or that holds GPU memory that cannot be released between calls, the boundary is where things get awkward. DeepHyper's model assumes evaluations are independent and restartable. Objectives with cross-evaluation state, such as a shared cache that must persist across trials, do not fit that assumption cleanly, and the README offers no guidance on the case.

A third point is simply that the search is Bayesian and therefore sequential in its proposals. The parallelism comes from having several evaluations in flight, not from the optimizer itself. With num_workers=2 the quickstart shows the incumbent updating at rows 1, 4 and 96, which is what a small worker pool looks like. Raising worker count is the lever, and the README does not discuss how CBO behaves when the pool is very large relative to max_evals.

DeepHyper against Optuna and Ray Tune

The closest comparison is Optuna, which also provides Bayesian search over a declarative search space and is the default choice for single-machine tuning. The difference is where the distribution lives. Optuna's storage-backed model lets separate processes or machines act as workers that pull trials from a shared database, but the trial itself is a Python callable running in the worker process. DeepHyper inverts this: the run function is bound to an Evaluator, and the evaluator is the component that knows how to place work. That inversion is what lets DeepHyper target MPI and scheduler environments, and it is also why adopting it means writing your objective to the job.parameters contract rather than to your own signature.

Ray Tune is the other reference point, and it takes a third approach: the tuning library owns the cluster. Ray expects to start and manage workers, which is natural on a Ray cluster and less natural on a machine where the batch system already decides what runs where. DeepHyper's evaluator abstraction does not require it to own the scheduler. For a team whose jobs are submitted through an existing allocation, that is the relevant distinction.

None of this makes DeepHyper the better library in general. It makes it the one whose assumptions match a specific environment. If your evaluations are already jobs, DeepHyper's contract is closer to your reality than a callable-based sampler's.

Maintenance, releases and the BSD-3-Clause terms

The release cadence visible in the material is active: 0.13.0 in December 2025, 0.13.1 two weeks later, 0.13.2 in early January 2026, with the last push to the default branch in March 2026. The project is not archived. It is published in the Journal of Open Source Software under DOI 10.21105/joss.07975, with a citation block naming Romain Egele, Prasanna Balaprakash, Gavin M. Wiggins and Brett Eiffert. A JOSS paper implies a reviewed software artifact at a point in time, not a support contract.

The version numbering is pre-1.0, and the 0.13.x sequence shows patch releases arriving within weeks of each other. That pattern is normal for a research-driven library and it means you should pin the version in your environment and read the release notes before moving. The material does not include a changelog, so what changed between 0.13.0 and 0.13.2 is not something I can state.

The licence is BSD-3-Clause, a permissive licence that permits use in closed-source and commercial settings and requires retention of the copyright notice and disclaimer. That is the extent of what I can say here. Whether your organization's redistribution or patent posture is satisfied by BSD-3-Clause is a question for your legal team, not for this article.

Who should pick it up, and the one thing to check first

DeepHyper earns its place when the evaluation is the expensive part and the scheduling is already solved by something outside Python. The evaluator boundary is the feature, not the tax, in that setting. Teams running neural architecture searches over allocations on an HPC system are the intended users, and the institutional acknowledgments in the README point the same direction.

It is the wrong tool for interactive tuning on a laptop. If your objective runs in seconds and your search space has a handful of dimensions, the process evaluator adds serialization and a worker pool to a problem that a single-process sampler handles directly, and you gain nothing from a search loop designed to dispatch across a queue.

The first thing to verify before committing is not the optimizer but the evaluator you actually intend to use. The quickstart demonstrates method="process" with num_workers, and the README does not document the other methods or their keyword arguments. Confirm in the documentation that a scheduler-backed evaluator exists for your environment, that its method_kwargs cover your submission parameters, and that your objective can be expressed as a function of job.parameters returning a single maximized scalar. If that last condition fails, the rest of the library does not apply.

Editorial conclusion

Adopt DeepHyper if your evaluations already run as batch jobs and you need a search loop that can dispatch them across workers without rewriting the training script. Do not adopt it for tuning a scikit-learn model on one laptop, where a single-process sampler is less machinery for the same result. Before committing, verify that the search you intend to use is documented for your job mix, and check the evaluator's method_kwargs against your scheduler, because the process evaluator in the quickstart is not the same thing as a queue-backed one.

Official sources

1. deephyper/deephyper on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases