jMetal: a Java framework for multi-objective metaheuristics

Seven sub-projects, and what each one is for

jMetal is a Java-based framework for multi-objective optimisation with metaheuristics, and the last stable version is 7.8. It is a Maven project, and the sub-project table in the documentation is the fastest way to understand it. jmetal-core holds the core classes. jmetal-algorithm holds the algorithm implementations. jmetal-problem holds the benchmark problems. jmetal-lab holds experimentation and visualisation. jmetal-parallel holds the parallel extensions. jmetal-auto holds auto-design and configuration. And jmetal-component holds component-based algorithms. The split is a real design decision rather than an accident of history, because a study of one algorithm on one problem should not drag in a visualisation layer, and the component module then provides a way to build algorithms out of reusable parts.

The auto sub-project is frozen, and where that functionality went

One note in the documentation changes what the framework is today. jMetal 7.6 is the last version in which jmetal-auto is updated, and from then on that sub-project is frozen, since its functionality is covered by the Evolver project, which is under active development. Evolver is described as a Java framework for the automated meta-optimization of multi-objective metaheuristics, which formulates their automatic configuration and design as a multi-objective optimization problem, which is a neat bit of recursion. So if you were planning to auto-configure your algorithm, that path now leaves the jMetal repository, and the sibling project takes it over. The related projects list also includes jMetalPy, which is jMetal in Python, and SAES, a Python library to analyse and compare the performance of multi-objective algorithms.

The Python scripts, and the two ways to look at a front

The scripts/ directory holds Python utilities for visualising the Pareto fronts produced by jMetal algorithms, working from any FUN.csv file the algorithms produce. There are two, and they answer different questions. plot_front.py produces a static matplotlib figure, a two or three dimensional scatter, with parallel coordinates for more than three objectives, saved as PNG, PDF or SVG. plot_front_interactive.py is an interactive Plotly viewer where you can rotate a three dimensional front and hover for values, opening in the browser or saving as HTML. Both are installed the same way, from the requirements file in the scripts directory, in a virtual environment or a conda environment, and the usage examples show that a reference front can be passed alongside for comparison.

    # Usage examples
    python scripts/plot_front.py FUN.csv
    python scripts/plot_front.py FUN.csv referenceFront.csv --mode side --output fig.png
    python scripts/plot_front_interactive.py FUN.csv referenceFront.csv --mode both

The changelog, and the fixes that matter for reproducibility

The changelog is where the real behaviour of the framework is documented, and the entries are unusually specific about the kind of bug that quietly changes results. One fix makes Spread and GeneralizedSpread sort local copies rather than the arrays they receive, because the reference front is shared by all the calls of an indicator instance, and sorting it in place from several threads at once corrupted it and made Arrays.sort fail with a comparison method violating its general contract error, which happens when you compute the indicators of independent runs in parallel. Another fixes the decision space of MaF08, whose variables had been bounded to [0, 1], a range containing only a fifth of the polygon that is its Pareto set, so most of its Pareto front could not be reached, and the bounds are now [-10000, 10000] as in the definition of the test suite. The consequence is stated plainly: results for MaF08 obtained with earlier versions are not comparable with the new ones.

The new three-objective reference fronts

The 7.8 release added reference fronts for the MaF problems with three objectives, from MaF01 to MaF15, alongside the existing ones for five, ten and fifteen objectives. They live in resources/referenceFrontsCSV as files named MaF with the objective count and a three-dimensional suffix, and they were written with the method PlatEMO uses for that benchmark, where points are spread on the simplex and mapped to the front of each problem. The generation script is scripts/maf_reference_fronts.py in the Evolver repository, and Evolver has an integration test called MaFReferenceFrontsIT that checks them against the problems in jMetal. That is a good sign about how the two projects relate: the reference data is produced elsewhere and verified here.

The smaller fixes, which are the ones that bite

Two more entries are worth reading because they are the kind of change that only shows up under load. The AngleDensityEstimator now returns 0.0, the worst density, for a solution whose density has not been computed, in the way the crowding, k nearest neighbour and shifted estimators do, instead of throwing an exception, because a selection that compares the population by density before the estimator has computed it, whether a tournament, a ranking or a stochastic universal sampling in an algorithm such as RDEMOEA or SSMOEA, used to fail with a null object parameter error. And the two JMetalException constructors that take a message and a cause, or just a cause, now keep both, where they used to only log the error, so the exception they created had neither, and whoever caught it got null instead of what went wrong. That last fix is a good example of the kind of change that never appears in a feature list.

Where jMetal fits, and what it does not replace

jMetal is for the specific problem of running multi-objective metaheuristics properly, which means a research workflow where an algorithm has to be compared against a reference front, the quality indicators have to be computed, and the result has to be reproducible. The framework covers all of that, and the reference front and indicator work is exactly the part that is easy to get wrong when you implement it yourself. What it is not is a general machine learning framework. There is no training loop, no autodiff, and no model zoo, because the problem domain is different, and the algorithms in jmetal-algorithm are the classical ones from that literature. It is also not a replacement for a modern auto-tuning toolchain, given the jmetal-auto freeze, and if your project is going to grow its own configuration search, that is now Evolver's territory rather than this repository's.

Editorial conclusion

jMetal fits a Java researcher or engineer who needs to run a real multi-objective algorithm against real benchmark problems and compare it properly, since the framework provides the algorithms, the problems, the indicators, the reference fronts and the visualisation as one coherent package, and it fits a project that only wants the core and the algorithms without dragging in the lab. It does not fit a team looking for a modern machine learning toolchain, because the framework is built around the classical Pareto-dominance machinery of the multi-objective literature rather than around neural training, and it does not fit a project that has already committed to an auto-tuning approach, because jmetal-auto is frozen as of jMetal 7.6 with its functionality moved to the Evolver project. Before you build on it, read the changelog rather than the tutorials, because that is where the fixes live, and check the reference fronts in resources/referenceFrontsCSV, since the new three-objective MaF fronts arrived in the 7.8 release alongside a change to the MaF08 decision space that makes older results incomparable.

Frequently asked questions

Official sources

1. Issues
2. jMetal/jMetal on GitHub
3. License: MIT
4. Project website
5. README