Holy Build Box gives your C binary an old glibc to link against

The glibc symbol problem, stated precisely

Holy Build Box is a system for building cross-distribution Linux binaries, and the README's own description is a system for building portable binaries for Linux, binaries that work on pretty much any Linux distribution, by providing an easy-to-use compilation environment with an old glibc version.

The problem it addresses is described at the top of the README. If you have ever built a C or C++ Linux binary, you have probably noticed that it may not run on other Linux distributions, or even other versions of the same distribution, and that this happens even if your application does not use newer APIs. The README contrasts this with Windows binaries, which tend to work on pretty much every Windows machine.

The primary cause is glibc symbol versioning. Each function in glibc has multiple versions, which lets the glibc developers change a function's behaviour without breaking applications that relied on bugs or implementation-specific behaviour. During linking, the linker helpfully links against the most recent version of the symbol. The result is the error message the README quotes:

./foo: /lib/libc.so.6: version `GLIBC_2.11' not found

The only way to tell the compiler and linker to use older symbol versions is linker scripts, and the README is blunt that this requires specifying the version for each and every symbol, which is an undoable task.

Holy Build Box solves this by supplying a tightly-controlled build environment that contains an old version of glibc, so the linker has no newer symbol to prefer. The whole project is that one idea plus the machinery to make static linking work.

Why static linking to glibc is the wrong answer

The README has a short section on why it does not advocate statically linking to glibc, and the reasoning matters for anyone who arrives at the same problem and considers the obvious workaround.

Statically linking to glibc wastes a lot of space. A simple 200 KB program can suddenly become 5 MB.

It also introduces runtime problems. Such binaries are not able to call dlopen(), so applications which want to load plugins at runtime are not compatible with this approach. For a project whose whole point is a self-contained binary, losing the ability to load plugins at runtime is a serious functional regression, and it is not something the size saving compensates for.

The third factor is security, and it is handled in the FAQ as its own question: static linking introduces security problems, and the README's position is that because glibc stays dynamically linked, the system library can be patched independently of your binary. A user running an old, unpatched glibc alongside your statically linked binary is still exposed to the glibc vulnerabilities, and only the dynamic link gives you the chance to fix that on the user's machine.

So the design is: old glibc, dynamically linked, for symbol compatibility and for patchability. Everything else, statically linked, so the dependency version does not matter.

The static linking strategy and its three problems

The README is careful about the scope of static linking. The note says the approach is to statically link to all dependencies, except for glibc and other system libraries that are found on pretty much every Linux distribution, such as libpthread and libm.

It also says plainly that this is not advocating static linking of everything, which is the usual way advice like this gets misread.

Static linking itself has three known problems, and the README lists all three. First, many build system tools are only designed with dynamic linking in mind and require many tweaks before they work with static linking, with the example that the default OpenSSL pkg-config entries do not work with static linking at all. Second, if the build environment is not tightly controlled, various build systems may pull in too many dependencies, and the example is sharp: if libssh happens to be installed on your workstation because some utility application needs it, a third-party application's build system will detect it and link to it, so to produce a binary without SSH functionality you now have to uninstall libssh and break the utility application you were using. Third, the static libraries provided by your distribution may not be compiled with the flags you want.

Holy Build Box solves problem two completely, by providing a tightly-controlled build environment that is isolated from your host environment, and this works through Docker, because Holy Build Box is a Docker image. It partially solves problems one and three by providing static versions of often-used libraries that tend to be a pain to set up for proper static linking.

The isolation is the point. You are not asked to tidy your workstation, you are asked to build inside a container where the library set is known and intentional.

The image, the architectures and how the build runs

The Dockerfile at the repository root shows the whole thing in four lines, and it explains a lot about the design.

FROM rockylinux:8
ADD image /hbb_build
ARG DISABLE_OPTIMIZATIONS=0
RUN bash /hbb_build/build.sh

The base is Rocky Linux 8, the image directory is copied in as /hbb_build, and a build script assembles the environment. The DISABLE_OPTIMIZATIONS build argument is threaded through, and it also appears in the Makefile as a variable, which is the same flag the test targets set to 1.

The Makefile shows how the images are produced and where they live. The version is 4.0.2, the image is named from the owner and project as phusion/holy-build-box, and the registry switches on the environment: in GitHub Actions the image goes to ghcr.io, otherwise to docker.io. The two architecture targets are separate docker buildx invocations, one for linux/amd64 and one for linux/arm64, each producing its own tag, and there is a matching set of test targets that run the same build script inside rockylinux:8 with the image directory mounted read-only, with SKIP_FINALIZE and DISABLE_OPTIMIZATIONS set.

That is a sensible arrangement for a project whose main deliverable is an image: the build is reproduced on both architectures in CI, the same script does the work, and the release tags are version-architecture pairs such as 4.0.2-amd64, with a major-version tag and a latest tag applied on top.

The repository also has a Vagrantfile and a dev directory, which suggests a local development path alongside the Docker workflow, and the README points to the project website for the getting started pages.

The documentation set, and the two questions worth reading first

The repository is mostly documentation, and the file names tell you what a user has to decide.

There are seven numbered tutorials: TUTORIAL-1-BASICS.md through TUTORIAL-7-VERIFYING-PORTABILITY-WITH-LIBCHECK.md. The last one is the one to read first if you are impatient, because verifying portability with libcheck is the step that tells you whether the artefact actually works, rather than assuming it does.

Around them sit the reference documents. ENVIRONMENT-STRUCTURE.md explains how the build environment is put together, ESSENTIAL-SYSTEM-LIBRARIES.md covers the libraries that stay dynamic, LIBRARY-VARIANTS.md covers the static library builds, CACHING-WITH-CCACHE.md covers build caching, LINKING-CXX.md covers the C++ linking case, and SECURING-THE-BUILD-ENVIRONMENT.md and SECURITY-HARDENING-BINARIES.md cover the two sides of the security story, the build host and the produced binary. INSTALLING-ADDITIONAL-DEPENDENCIES.md and LIBCURL-SSL-CERTIFICATE-AUTHORITIES.md cover the two things that most often go wrong in practice, an extra library you need and the certificate authorities a bundled libcurl will not find on the user's machine.

The FAQ is worth reading before the tutorials, because two of its questions frame the decision. The first is who should be interested in portable Linux binaries, which is a scoping question rather than a technical one. The second is how Holy Build Box compares to using Docker to package up an application, which is the comparison most people actually arrive with.

There is also a Changelog.md, a CONTRIBUTING.md, a CODE_OF_CONDUCT.md and a LICENSE.md, and the README carries a maintainers wanted badge, which is a useful signal about the state of the project.

Where Holy Build Box is the wrong tool

Three limits, and the first one is the most common misapplication of the idea.

The first is that it does not apply to interpreted applications. The README has a FAQ entry specifically about how to deal with interpreted applications such as ones written in Ruby, Python or Node.js, and the answer is that this approach does not apply to them. An interpreted application does not have a glibc symbol problem to solve, because there is no linked binary. If your product is a Python service, a Ruby gem or a Node package, the portability problem is a packaging problem and a runtime dependency problem, and Holy Build Box is the wrong instrument.

The second is that a statically linked binary is not automatically a good binary. The README has its own caveats section and the FAQ asks whether the tool is suitable for all applications. A binary with everything statically linked is larger, and static linking has its own consequences for updates, for behaviour differences between the bundled library and the system one, and for the debugging experience. There is also a question of whether a large static binary is what your users want to download when a package manager would serve them a patched build.

The third is the age of the glibc. The whole mechanism depends on the build environment containing an old glibc, which means your binary will be pinned to symbol versions from that era. That is the point for compatibility, and it is also a constraint on what you can link against, and it means the build environment has to be maintained as distributions move. The repository's own state, with a maintainers wanted badge and no GitHub releases, is a fair signal that this is a project you should evaluate carefully before depending on.

The alternative approach, and it is the one most Go projects take for free, is the comparison the README makes: a language like Go produces a static binary with no dynamic library dependency at all, so the portability question never arises. The difference in approach is that Go solves it by having no glibc dependency in the first place, while Holy Build Box solves it for code that has to link against glibc by constraining which glibc it links against, and it is the only option for a C or C++ codebase that must keep its current toolchain and dependencies.

Editorial conclusion

Adopt Holy Build Box if you ship a C or C++ binary to users on many Linux distributions and have been chasing glibc symbol version errors, because the controlled old-glibc environment plus static linking of dependencies solves exactly that and the project is explicit about the trade-offs. Do not adopt it for interpreted applications in Ruby, Python or Node.js, which the README addresses separately, and do not adopt it if you need a binary that picks up security fixes from a patched system glibc at runtime. Verify first that the older glibc symbol versions you depend on are still present in the image, that your build system can be made to statically link the dependencies you ship, and that the image variant you pull matches the architecture you target, since the Makefile produces separate amd64 and arm64 images and the repository records a version of 4.0.2.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. phusion/holy-build-box on GitHub
4. Project website
5. README