libMesh builds a finite element library around a choose-your-methods configure step

A finite element library, and a build model that is the interesting part

libMesh is a C++ finite element library, and its repository topics name the domain precisely: amr, fem, finite-element-analysis, finite-element-methods, finite-elements, hacktoberfest, parallel and simulation. The homepage is libmesh.github.io, the licence is LGPL-2.1, and the default branch is devel rather than master.

What makes libMesh unusual is not the finite element part, which is the well established part of the project, but the build system around it. The library supports the notion of multiple methods, which are configuration settings used to build the library, and the README documents five of them.

opt is the fully optimized mode, with little to no error checking and no debugging symbols in the resulting library, using aggressive optimization flags. dbg is the full debugging mode, with all useful compiler warnings enabled and strict internal state checking, at the cost of allowing the asymptotic complexity of some algorithms to be worse than the design spec states. devel is the middle ground, with high levels of compiler optimization and internal state checking enabled, with debugging symbols included, though the README notes the resulting code is not always easy to navigate in a debugger because of the compiler optimizations. pro is an optimized code path with compiler flags suitable for use with gprof, and oprof is the equivalent for oprofile.

That is a real piece of design. A single source tree can be built as a fast library with no safety net, a slow library with a full one, or a build instrumented for a profiler, and the choice is a configure flag rather than a set of preprocessor defines scattered through the code.

devel, master and release tarballs, and the archive you must not download

The branch status section explains how the project moves, and it matters for anyone tracking a specific version.

Pull requests merge new code into devel, and the master git branch is automatically synced to each new devel commit that passes extended testing. Releases are created from branches from master, and also receive backported bugfix commits from subsequent development.

So there are three states of the code at any time. devel is where work lands, with extended testing attached to it. master is the last commit that passed the extended testing, and it is the branch releases are cut from. And a release is master plus backported fixes, which is why a release tarball and master can differ.

The extended testing status is tracked with a build status badge, and the repository carries a .coverage file and a fsanitize_ignorelist.txt at the root, which is consistent with a sanitiser build being part of the test matrix.

Then there is the warning, and it is the most practically useful paragraph in the README. Do not download a GitHub-generated ZIP archive. These do not contain the required submodules, and therefore cannot be used to build libmesh. Use only git clones or release tarballs when following these instructions.

The reason is the .gitmodules file at the root of the repository. A ZIP download flattens the working tree and drops the submodule checkouts, and a build that expects third-party sources in a specific place fails in a way that is tedious to diagnose. The release tarballs are prepared differently and do carry what the build needs.

Out-of-tree builds: configure, make, check, install

The build instructions are a numbered list, and the design choice in them is the first line. The default is to build libMesh out of tree, that is, within a separate build directory, rather than in the source tree itself, and the README gives the reason: this simplifies the process of having multiple, independently-configured builds.

The steps are these. Go to the location of the libMesh clone or extracted release tarball. If you are using a git clone, run git submodule update --init --recursive. Then mkdir build, cd build, and optionally run ../configure --help | less to see the configuration options. Then run ../configure --prefix=/some/final/install/directory, then make, then optionally make check, which runs the example programs and unit tests when possible, and then make install.

Afterward, make uninstall is available if needed to remove the files copied to the install directory and its subdirectories by make install.

Two details are worth carrying into your own build. The configure step is run from the build directory using a relative path, ../configure, which is what keeps the source tree clean. And the check target is explicitly optional, which tells you the examples and unit tests are not always runnable in every environment, since some of them need the optional packages discussed later.

The repository listing shows the autotools machinery behind this: configure and configure.ac, aclocal.m4, Makefile.am and Makefile.in, a bootstrap script, build-aux, m4 for the macros, and a configure script that is checked in. The build you invoke is the familiar GNU autotools one, configured for a C++ library with a C++11 requirement.

Selecting methods, and what the default buys you

Choosing which methods to build is a configure option, and the README gives two ways to pass them.

One is on the command line:

    ../configure --with-methods="opt dbg devel"

The other is as a variable:

    ../configure METHODS="devel oprof"

And if you say nothing at all, the default is METHODS="opt dbg devel", which is the same set as the first example.

The default is worth understanding, because it is a deliberate choice rather than an accident. Building opt, dbg and devel together gives you a library you can ship in production, one you can run the test suite against with internal state checking on, and one you can develop against, all from the same source tree and the same build directory. The internal state checking is the expensive part, and having it compiled out in the optimised build is what keeps that build usable for large problems.

The devel method is the one to understand next, because it is the unusual choice. It keeps optimisation on while enabling internal state checking, so the code still runs at close to production speed but validates its own invariants, and it ships with debugging symbols even though the optimisation makes the debugger experience awkward. That is a defensible trade for a library where a silent invariant violation costs a day of simulation time.

The pro and oprof methods exist for the other half of the problem. If you need to find out where the time goes, you need the code compiled with the flags your profiler expects, and choosing the method is how you get them without maintaining a separate branch.

One source tree, several compilers, side by side

The multiple builds section shows the out-of-tree design paying off, and the example is a module system workflow that many HPC sites will recognise.

The README says libMesh fully supports out-of-tree builds, and that users are encouraged to use this feature when needed to support multiple compilers. The example shares a single source tree and creates a subdirectory per compiler build:

    export LIBMESH_SRC=/local/libmesh
    cd $LIBMESH_SRC
    module load gcc/9.3
    cd $LIBMESH_SRC && mkdir gcc-9.3 && cd gcc-9.3 && ../configure && make && make install
    module swap gcc/9.3 intel/19.0
    cd $LIBMESH_SRC && mkdir intel-19.0 && cd intel-19.0 && ../configure && make && make install

The pattern is simple and worth copying for any autotools project. The source tree is set once in an environment variable, each compiler gets its own build directory, and the configure invocation is relative so it operates on the shared source. The installed copies can then be distinguished by the compiler that produced them, which is what you want on a cluster where a site supports two compilers and the ABI differs between them.

The same section also hints at the maintenance structure. The repository carries a Makefile.am and a Makefile.in at the root for the library itself, and an examples directory with its own Makefile.am and a run_common.sh, so the examples build with the same out-of-tree discipline as the library.

Dependencies are minimal, and the optional packages are where the work is

The dependency statement is short and worth reading carefully.

libMesh has no required dependencies other than a C++ compiler which fully supports the C++11 standard. To run on distributed memory platforms in parallel, you will also need MPI.

That is the whole mandatory list. A C++11 compiler and, for parallel runs, an MPI implementation.

The optional packages are the interesting part. The README says a large number are supported, with some distributed inside the contrib directory and compiled directly with libMesh, which are usually enabled by default, and others usable via third-party installations. A third party package like PETSc typically is only enabled if a corresponding environment variable, such as $PETSC_DIR and $PETSC_ARCH, or a command line option is used at configure time.

So there are two categories of optional dependency, and they behave differently. The bundled ones in contrib come along unless you turn them off. The third-party ones have to be pointed at, and the environment variable convention with a directory and an architecture suffix is the usual autotools pattern for finding an installation that is not on the default search path.

The repository listing backs this up with a large contrib directory, a doc directory, an examples directory with categories such as adaptivity, adjoints, eigenproblems, fem_system, finite_volume, introduction, miscellaneous, optimization, reduced_basis, solution_transfer, subdomains, systems_of_equations, transient and vector_fe, plus reference_elements, plot_shape.py, and a tests directory.

The dependency position is a real advantage for a numerical library. A solver library that requires a stack of third-party packages to build is a library that gets abandoned, and the combination of a self-contained core and opt-in external packages is why this one can still be built on a machine with nothing but a compiler.

Editorial conclusion

Simulation teams that need a configurable finite element code should look at libMesh, because the methods model lets one source tree produce an optimised library, a fully checked debug build or a profiled build without maintaining separate branches, and the out-of-tree build makes multiple compiler configurations cheap. It is the wrong choice if you want a single prebuilt binary with no configure step, and it is the wrong choice for a project that cannot take a C++11 dependency plus optional solver packages. Before building, read the warning about GitHub ZIP archives, because they omit the submodules the build needs, and decide which methods you actually want rather than accepting the default of opt, dbg and devel.

Frequently asked questions

Official sources

1. libMesh/libmesh on GitHub
2. License: LGPL-2.1
3. Project website
4. README
5. Releases