bulk_extractor 2.2.0: recursive scanning for digital evidence

Why bulk_extractor examines raw bytes

bulk_extractor addresses a specific digital-forensics problem: structured data may exist inside content that a conventional filesystem or file-format parser does not expose as a normal file. Its documented inputs include disk images, individual files, and directories. Its documented outputs include email addresses, credit card numbers, JPEG data, JSON snippets, and text files suitable for inspection or later forensic processing.

The project deliberately does not parse filesystem structures. That choice makes it useful during triage, when an analyst wants to examine the available byte stream rather than depend on a complete filesystem hierarchy or on each embedded format being recognized by a traditional carver. It also creates an important limitation: a result indicates that a scanner found a feature in the input, not that the surrounding filesystem metadata or original file context has been validated.

The intended users are digital-forensics investigators and researchers who need to collect many candidate artifacts from large or irregular inputs. The README also identifies investigative and law-enforcement applications for feature histograms, including histograms of email addresses and Google search terms. Those histograms are auxiliary analysis outputs, not substitutes for examining the extracted evidence and its context.

Recursive decoding in bulk_extractor 2.2.0

bulk_extractor's distinguishing mechanism is byte-level probing. According to the README, it checks every byte to determine whether that position can begin a sequence that can be decompressed or otherwise decoded. When decoding succeeds, bulk_extractor recursively examines the decoded data. This lets it discover objects such as BASE64-encoded JPEGs and compressed JSON objects that traditional carving tools may miss.

This design differs from starting with a filesystem and walking known files. A disk image, directory, or individual file is treated primarily as a stream of bytes to which scanners are applied. Scanner modules can emit structured findings, which are then placed in inspectable text output. The repository also supports file-based histograms of discovered feature classes.

The source is primarily C++ and the current release requires C++17. Scan modules are loadable shared libraries named `scan_*.so` on Linux-like systems, `scan_*.dylib` on macOS, or `scan_*.dll` on Windows. A module exports this C-linkage factory:

extern "C" scanner_t *bulk_extractor_scanner_v1();

The factory returns a normal `scanner_t` function. Scanner modules must be built against the same bulk_extractor source version as the executable, and their `PHASE_INIT` handler must call `sp.check_version()`. That ABI and version requirement make custom scanner development a source-level integration task rather than a plug-and-play extension.

Building the source tree and validating 2.2.0

The repository describes itself as the development tree and directs production users toward tested release downloads. It builds version 2.2.0, so a source build is most appropriate when development, custom scanners, or local platform verification are required. The README's short build sequence is:

git clone https://github.com/simsong/bulk_extractor.git
cd bulk_extractor
./bootstrap.sh
./configure
make
make check
make install

`bootstrap.sh` prepares the source build, `./configure` generates the build configuration, `make` compiles the project, `make check` runs the project's checks, and `make install` installs the built software. Platform package preparation may still be required, so `doc/installation.md` is the source-controlled installation guide.

Current validation covers Apple Silicon macOS and Ubuntu 22.04, along with current macOS GitHub Actions runners. Debian 10 is explicitly unsupported for native builds. The README also warns that older preparation scripts under `etc/` are not equivalent to current CI support and may need maintenance.

The README excerpt does not provide a complete first-scan command with input and output arguments, so supplying one would require assuming flags that are not documented here. The published operating manual at `https://simsong.github.io/bulk_extractor/` is the documented place to verify the invocation before scanning evidence. This missing command in the project overview is a documentation boundary, not evidence that the scanner lacks a command-line interface.

Container scanning and its deliberate omissions

The repository includes a multi-stage Debian Bookworm `Dockerfile` for reproducible Linux scans of regular image files. Its build stage installs the C++ build dependencies, copies the source, runs `bootstrap.sh`, configures with libewf disabled, and builds with two jobs. The runtime stage installs the required libraries, creates an unprivileged user with UID 10001, copies the executable, and sets `bulk_extractor` as the container entry point.

The documented container boundary matters. Input should be mounted read-only, while the output directory must be writable. The image runs without root privileges and intentionally omits `libewf` and Lightgrep. It is not designed as a privileged appliance that reads raw devices directly. That omission also means the Docker configuration is narrower than some full source builds, especially where E01 support matters.

The Windows CI path is narrower still. Native Windows builds are not currently supported. A GitHub Actions workflow cross-compiles `bulk_extractor64.exe` on Ubuntu and uploads it as `bulk_extractor-windows-x86_64`. The workflow includes a static libewf source release and validates the PE executable, but the artifact is unsigned and is not a release installer. The README states that the CI artifact includes E01 support, making it more than a plain command wrapper, but it does not turn Windows into a currently supported native build target.

Scanner controls, failure modes, and platform limits

bulk_extractor exposes several debugging environment variables that can alter execution. `DEBUG_BENCHMARK` adds CPU benchmark information to `report.xml`. `DEBUG_NO_SCANNER_BYPASS` disables logic that skips scanners for some `sbuf` objects based on ngrams or distinct-character counts. `DEBUG_HISTOGRAMS` prints file-based histogram debugging, while `DEBUG_HISTOGRAMS_NO_INCREMENTAL` disables incremental memory-based histograms. `DEBUG_PRINT_STEPS`, `DEBUG_SCANNER_DUMP_DATA`, and `DEBUG_SCANNERS_IGNORE` support more detailed inspection.

These controls are operationally significant. Disabling scanner bypass changes which scanners receive input, while turning off incremental histograms changes histogram behavior. Debug settings can therefore change observable output or runtime work. The README also suggests `-xall` for a run without scanners and a random sample of 0.001% for debugging image-size reading and seeks, but it does not give a full command line for either case in the README.

The clearest failure case is platform mismatch. Debian 10 is unsupported for native builds, older `etc/` preparation scripts may not reflect current CI, and Windows artifacts are cross-compiled, unsigned CI outputs rather than native supported builds. Raw-device scanning is also outside the documented Docker image's scope. Custom loadable scanners add another integration boundary because source-version compatibility and `sp.check_version()` are mandatory.

Maintenance, release use, and license handling

The repository is not archived, and its last push was on 2026-10-12. Release history includes `v2.2.0` on 2026-08-18, `v2.1.1` on 2024-04-27, and `v2.1.0` on 2024-01-25. The source tree currently builds 2.2.0, while the README explicitly prefers a tested release for production use.

That distinction affects upgrade planning. A source build can be useful for custom scanners, development, or environments where a tested release is unavailable. It also transfers more responsibility to the operator for dependencies, platform preparation, checks, and version consistency. Scanner modules built against a different source version are outside the documented compatibility contract.

The project's `LICENSE.md` distinguishes original U.S. government work from later contributions: it states that post-NPS project-authored code and documentation are GPL-3.0-or-later, while third-party components retain their own terms. Review the licence file, source notices and dependency licences for the exact release and distribution you use. Do not treat the entire project as public domain.

Editorial conclusion

Adopt bulk_extractor when investigators need broad extraction from files or regular disk images and can accept a byte-oriented workflow that does not parse filesystem structures. Do not choose it as the sole Windows tool or as a privileged raw-device appliance. Verify the exact scanner configuration, output location, license terms in `LICENSE.md`, and whether your inputs match the supported image formats before processing evidence.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. simsong/bulk_extractor on GitHub