# crash reads kernel core dumps and live systems through an embedded gdb

> The crash utility is a self-contained core analysis suite for the Linux kernel, built around an embedded gdb module that understands kernel data structures across versions. It reads kdump, kvmdump, xendump, netdump, diskdump, LKCD and mcore formats, and the README makes the build, cross compilation and the -g requirement unusually explicit.

**crash-utility/crash** — Linux kernel crash utility NOTE: subscribe to mail list via https://lists.crash-utility.osci.io/admin/lists/devel.lists.crash-utility.osci.io/ for contribution and discussion. github pull requests will be reviewed but it is not the main development channel.

- Repository: https://github.com/crash-utility/crash
- Website: https://crash-utility.github.io
- Stars: 968 · Forks: 316
- Language: C
- License: not declared
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/crash-utility-crash

## A core analysis suite built for kernel developers

The crash utility answers a specific question: what was the kernel doing when it died, or what is it doing right now. The README describes it as a self-contained tool for investigating either live systems or kernel core dumps, and the audience follows from that. This is not a general purpose debugger for user space programs. It is a tool for people who read kernel source, and the commands it offers, context specific stack traces, source code disassembly, kernel variable displays, memory display and dumps of linked lists, are written on the assumption that the reader knows what they are looking at.

The lineage is stated in the README. The tool is loosely based on the SVR4 crash command, but it has been completely integrated with gdb in order to display formatted kernel data structures and disassemble source code. That integration is the defining feature rather than an add-on. Any gdb command may be entered and it is passed to the gdb module for execution, so the tool is a specialised front end sitting on a real debugger rather than a debugger of its own.

There is also an extensibility angle that matters for kernel developers. Several commands delve deeper into specific kernel subsystems, and the README describes these as templates for kernel developers to create new commands for analysis of a specific area of interest. Adding a new command is described as a simple affair, and a quick recompile adds it to the command menu.

The documentation is unusual in a good way. A whitepaper with complete documentation about the use of the utility is hosted at crash-utility.github.io/crash_whitepaper.html, and the project has its own site at crash-utility.github.io.

## One binary reads a long list of dump formats

The list of formats the tool can read is the reason it has survived for so long. The README names kernel core dumps created by dump creation facilities such as kdump, kvmdump and xendump, the netdump and diskdump packages offered by Red Hat, the LKCD kernel patch, the mcore kernel patch created by Mission Critical Linux, as well as other formats created by manufacturer specific firmware.

Look at the repository layout and the same story appears in the file names. There is kvmdump.c with its header, diskdump.c, lkcd_common.c and a series of lkcd_v1, lkcd_v2_v3, lkcd_v5, lkcd_v7 and lkcd_v8 source files that correspond to the LKCD dump format revisions. There is memory_driver/ for live system access, filesys.c, memory.c, kernel.c and global_data.c.

The consequence for a user is that the acquisition method is not the thing you have to agree on. Whether the dump came from kdump on your own hardware, from a diskdump image of a crashed appliance, or from firmware on a vendor system, the same binary opens it.

Architecture coverage is listed in the same spirit. x86, ia64, x86_64, ppc64, ppc, arm, arm64, alpha, mips, mips64, loongarch64, riscv64, s390 and s390x based kernels are supported, with other architectures possibly addressed in the future. The per architecture source files in the repository root, alpha.c, arm.c, arm64.c, loongarch64.c, mips.c and the rest, match that list.

## Building means a tarball, a make, and a few minutes for gdb

The build instructions in the README are short and complete, which is a change from the usual archaeology involved in an old C project. Replace VERSION with the actual release filename and extracted directory, then unpack and build:

```bash
tar -xf crash-VERSION.tar.gz
cd crash-VERSION
make
```

By default, debug information is not built into either crash or the embedded gdb module. To build both with debug information:

```bash
make DEBUG=1
```

A summary of the available make targets and build options is printed by:

```bash
make help
```

The build dependencies are listed per distribution, and they are not light. The Fedora line asks for gcc, gcc-c++, ncurses-devel, zlib-devel, lzo-devel, snappy-devel, bison, wget, patch, texinfo, libzstd-devel, gmp-devel and mpfr-devel, while the Ubuntu and Debian line asks for gcc, g++, libncurses-dev, zlib1g-dev, liblzo2-dev, libsnappy-dev, bison, wget, patch, texinfo and libzstd-dev. Those are the gdb build requirements arriving through the embedded module.

The README warns that the initial build takes several minutes because the embedded gdb module has to be configured and built, and offers an alternative route: a crash source RPM file may be installed and built, and the resulting crash binary RPM installed.

Two details in the build hygiene are worth copying. To start over, make distclean removes every build product, the generated configuration, the extracted gdb source directory and any release tarball, spec or source RPM present, while plain make clean keeps the configured gdb directory so that rebuilding after editing crash sources stays fast. A build never modifies the distributed Makefile, and the configured values are kept in a generated Makefile.inc file.

## Cross compilation and the 32-bit binary matrix

A debugger that reads dumps has an awkward property: the binary has to understand the architecture of the dump, and often it has to be the same architecture as the host. The README is explicit that the crash binary can only be used on systems of the same architecture as the host build system, and then it enumerates the exceptions, where a 32-bit binary built on a 64-bit host can analyse 32-bit dumps.

Those combinations are built with the target variable. On an x86_64 host, a 32-bit x86 binary for 32-bit x86 dumpfiles is made with make target=X86, and a 32-bit arm binary for 32-bit arm dumpfiles with make target=ARM. The same pattern covers make target=MIPS, make target=PPC on a ppc64 host, make target=ARM64, make target=PPC64, make target=RISCV64 and make target=LOONGARCH64. An arm64 host builds an x86_64 binary with make target=X86_64.

For a different host architecture entirely, the tool cross compiles. The README gives the example:

```bash
    $ make CROSS_COMPILE=aarch64-linux-gnu- -j`nproc`
```

with the supported cross compilation targets listed as x86_64, x86, aarch64, s390x, powerpc64, alpha, sparc64, mips and riscv64.

The Makefile in the repository root shows the same thing in the form the build system actually reads. The architecture is derived from the cross compile prefix when one is given, otherwise from uname -m, and then normalised:

```
ARCH := $(shell echo $(ARCH) | sed -e s/i.86/i386/ -e s/sun4u/sparc64/ -e s/arm.*/arm/ -e s/sa110/arm/)
```

The header of that Makefile also records the supported targets as X86, ALPHA, PPC, IA64, PPC64 and SPARC64, with TARGET and GDB_CONF_FLAGS configured automatically by the configure step. The practical upshot is that the matrix is not complete, and a team that supports an unusual architecture should check the list before promising coverage.

## Without a vmlinux built with -g, the analysis stops early

The hardest requirement in the README is not about building crash, it is about the kernel you want to analyse. To contain debugging data, the top level kernel Makefile CFLAGS definition must contain the -g flag. Typically a distribution will contain a package with a vmlinux file holding full debuginfo data, and if not, the kernel has to be rebuilt.

The README gives the exact edits for the old kernel series. For 2.2 kernels, the CFLAGS line changes to:

```
        CFLAGS = -g -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
```

For 2.6 and later kernels the kernel should be configured with CONFIG_DEBUG_INFO enabled, which adds the -g flag to the CFLAGS setting in the kernel Makefile. After the kernel is recompiled, the uncompressed vmlinux created in the top level kernel build directory has to be saved.

That saved vmlinux is the key that unlocks the rest of the tool. It is what lets the embedded gdb map kernel data structures, symbolise stack traces and resolve kernel variables, and the README's design goal statement explains why the tool bothers with all of this: the intent is to make it independent of Linux version dependencies, building in recognition of major kernel code changes so as to adapt to new kernel versions while maintaining backwards compatibility.

The backwards compatibility claim is broad. The utility can be run on any Linux kernel version dating back to 2.2.5-15, and maintaining backwards compatibility is named as a primary design goal. That is the property that makes crash usable across an estate of mixed kernels rather than only on the one you just built.

## Compressed dumps: libz by default, and the rest if makedumpfile used it

Compression is where the tool meets the acquisition tooling, and the README describes a specific dependency.

Traditionally, when vmcore files are compressed via the makedumpfile facility, the libz compression library is used, and by default the crash utility only supports libz. The README then notes that makedumpfile has been enhanced to optionally use LZO, snappy or zstd compression, and that crash has been extended to match, with the build dependency list accordingly including lzo, snappy and zstd development packages on every distribution.

The repository shows the same split in the source tree. There is lzorle_decompress.c with its header for the LZO variant, and the Makefile dependencies mirror the README. A separate file, makedumpfile.c with makedumpfile.h, covers the dump format itself, and bpf.c sits alongside it.

The practical rule for an analyst is straightforward. If the dump was produced with the default settings, crash can read it. If the acquisition side used one of the newer compressors, the matching support has to be present in the crash build, which is why those libraries appear in the dependency list rather than being optional extras.

There is also a packaging path for distribution maintainers. To create a source RPM from the tree, you run make release as root, which creates the crash-<version>.tar.gz source tarball, and then make srpm, which creates the crash.spec file and the source RPM from that tarball.

## Development happens on the mailing list, not on GitHub

The project description carries an unusual warning that anyone contributing needs to read. It says to subscribe to the mail list for contribution and discussion, and that GitHub pull requests will be reviewed but are not the main development channel.

That is a deliberate choice about how a project of this age and this kind of audience works, and it has consequences. A kernel developer who wants to contribute will do better by joining the list and discussing the change there first, and a team evaluating the project should not treat the GitHub repository as the place where support happens.

The release record shows a steady cadence. The recent releases are 9.0.3 on 2026-09-10, 9.0.2 on 2026-04-24 and 9.0.1 on 2025-11-20, and the last push to the repository was on 2026-10-10. The version is carried in the crash-release file at the root, and the tarball naming, crash-9.0.x.tar.gz, matches the release tags.

On the licence, the repository field is listed as unknown, while the header of the Makefile carries the GNU General Public License version 2 or later notice with the copyright lines for Mission Critical Linux and Red Hat, and a COPYING3 file sits in the root. For anyone shipping crash in a product, the licence text in the source tree is the thing to read, since the repository metadata does not settle the question.

## Conclusion

Adopt crash if you are a kernel developer or a systems engineer who has to answer questions about a live kernel or a core dump, and who needs kernel aware commands rather than a raw debugger. Do not adopt it to read a userspace core file, where a plain gdb session is the smaller tool, and do not build it from source on a machine that cannot satisfy the build dependencies, since the embedded gdb module has to be configured and compiled. Before you build, confirm that a vmlinux with full debug information exists for the kernel you intend to analyse, read the whitepaper linked from the README for the full usage documentation, and subscribe to the crash-utility devel mailing list, because the project states that GitHub pull requests are reviewed but are not the main development channel.

## FAQ

### What dump formats can the crash utility read?

Kernel core dumps from kdump, kvmdump and xendump, the Red Hat netdump and diskdump packages, the LKCD kernel patch, the mcore kernel patch, and other formats created by manufacturer specific firmware.

### How do I build the crash utility from source?

Unpack the release tarball, enter the directory and run make. The initial build takes several minutes because the embedded gdb module has to be configured and built, and the distribution build dependencies include gcc, ncurses, zlib, lzo, snappy, zstd and bison development packages.

### What do I need before I can analyse a kernel core dump?

A vmlinux with full debug information for the kernel, which usually comes from a distribution package or from a kernel rebuilt with the -g flag or with CONFIG_DEBUG_INFO enabled. The uncompressed vmlinux from the kernel build directory has to be saved for the analysis.

### Which architectures does crash support?

x86, ia64, x86_64, ppc64, ppc, arm, arm64, alpha, mips, mips64, loongarch64, riscv64, s390 and s390x based kernels are supported, and the README says other architectures may be addressed in the future.

### How do I cross compile crash for another architecture?

Pass a cross compile prefix to make, for example make CROSS_COMPILE=aarch64-linux-gnu- -j`nproc`. The supported cross compilation targets are x86_64, x86, aarch64, s390x, powerpc64, alpha, sparc64, mips and riscv64.

## Sources

- [crash-utility/crash on GitHub](https://github.com/crash-utility/crash)
- [Issues](https://github.com/crash-utility/crash/issues)
- [Project website](https://crash-utility.github.io)
- [README](https://github.com/crash-utility/crash/blob/master/README.md)
- [Releases](https://github.com/crash-utility/crash/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/crash-utility-crash