# HDiffPatch: small binary deltas and the memory bill behind them

> HDiffPatch is a C++ library and a pair of command line tools that diff and patch binary files or whole directories, with two diff modes that trade memory against patch size. It fits OTA update pipelines, and it is the wrong tool for signed Android packages and for pull-based updates.

**sisong/HDiffPatch** — a C\C++ library and command-line tools for Diff & Patch between binary files or directories(folder); cross-platform; runs fast; create small delta/differential; support large files and limit memory requires when diff & patch.

- Repository: https://github.com/sisong/HDiffPatch
- Stars: 2,123 · Forks: 351
- Language: C++
- License: NOASSERTION
- Published: 2026-10-14 · Updated: 2026-10-14 · Language: en
- Canonical page: https://hysenlabs.com/projects/sisong-hdiffpatch

## Two executables and a patch format of its own

HDiffPatch is a C and C++ library with two command line tools attached, hdiffz and hpatchz, that build and apply deltas between binary files or whole directories. The problem it addresses is narrow: when a device has to receive a new version of a large binary over a slow or metered link, sending the whole file again wastes bandwidth, while a line-oriented diff tool will mangle a firmware image. This one treats both sides as byte streams, locates matching blocks, and writes the difference plus the instructions for rebuilding it into a single file.

Every path argument can name a file or a directory, so one three-argument command covers both a single image and a whole application tree. Its own patch format is the default, though the library also reads patches written in the bsdiff4 and endsley/bsdiff formats, and patches in VCDIFF, the encoding specified in RFC 3284 and used by xdelta3 and open-vcdiff. Consuming foreign patches matters operationally: a team can standardise on one patcher in the field while different build machines keep producing deltas with whatever tool they already have.

That compatibility is layered on rather than folded in. The repository keeps it in bsdiff_wrapper/ and vcdiff_wrapper/ beside the core, and the Makefile switches each one off independently through the bsdiff and VCDIFF toggles, so a product that never ships a VCDIFF patch does not carry that reader at all.

## Match in memory or as a stream, and what each one costs

hdiffz exposes two diff modes, and the choice between them is a memory decision rather than a speed decision. The default, `-m[-matchScore]`, loads everything into memory to get the smallest possible patch file, at a cost of the new file size plus five times the old file size, or nine times once the old file is at least 2GB, and a constant on top. The match score defaults to -m-4, with 0 to 4 suggested for binary content and 4 to 9 for text.

The alternative, `-s[-matchBlockSize]`, keeps both inputs as streams and asks for a far smaller working set: roughly oldFileSize*16/matchBlockSize plus matchBlockSize*5 for each parallel thread. Its default block size is -s-64, and 16, 32, 48, 1k, 64k or 1m are offered as alternatives. Patches produced this way still apply on several threads at once, which you ask for by adding -p-5 to the hpatchz command line, though the README says multi-threading is not recommended on a hard disk.

The option list as it stands in the README, third entry included:

```bash
  -m[-matchScore]
      DEFAULT; all file load into Memory; best diffFileSize;
      requires (newFileSize+ oldFileSize*5(or *9 when oldFileSize>=2GB))+O(1)
        bytes of memory;
      matchScore>=0, DEFAULT -m-4, recommended bin: 0--4 text: 4--9 etc...
  -s[-matchBlockSize]
      all file load as Stream; fast;
      requires O(oldFileSize*16/matchBlockSize+matchBlockSize*5*parallelThreadNumber)bytes of memory;
      matchBlockSize>=4, DEFAULT -s-64, recommended 16,32,48,1k,64k,1m etc...
  -block-fastMatchBlockSize
```

A third option, `-block-fastMatchBlockSize`, appears with no description at all, so its effect and its accepted range are not documented. Beyond these three lines, the README says nothing else about the option set.

## Building from source: two repository URLs in one README

Building is one command in the happy path, yet the README hands you two different clone URLs and never explains the difference. The short form targets HDiffPatch5.git and pulls submodules with it:

```bash
git clone --recurse-submodules https://github.com/sisong/HDiffPatch5.git <code_dir>
cd <code_dir>
```

The long form drops the version suffix and hands you eight clone commands, one repository per line:

```bash
cd <code_dir>
git clone https://github.com/sisong/HDiffPatch.git  HDiffPatch
git clone https://github.com/sisong/libmd5.git  libmd5
git clone https://github.com/sisong/xxHash.git  xxHash
git clone https://github.com/sisong/lzma.git  lzma
git clone https://github.com/sisong/zstd.git  zstd
git clone https://github.com/sisong/zlib.git  zlib
git clone https://github.com/sisong/libdeflate.git  libdeflate
git clone https://github.com/sisong/bzip2.git  bzip2
```

No line of the README says what that 5 stands for, so check what the binary reports before you ship a patch produced by it. Inside the tree, Linux and macOS need one command:

```bash
cd HDiffPatch
make -j
```

Work on the master branch reached v5.1.3 on 2026-07-31, five days after v5.1.2 on 2026-07-26 and nine days after v5.1.1 on 2026-07-22.

## Directories diff as bytes, and the metadata that never travels

Every path argument to hdiffz and hpatchz can be a file or a directory, which is how an entire application tree gets updated in one operation. Two further modes describe that tree in a separate file instead of walking it: `-M-old#oldManifestFile -M-new#newManifestFile` diffs two manifests rather than two directory trees, and a manifest can be written out through the `-M#outManifestTxtFile` option.

Checking work before it ships is cheap. The `-info` option prints what a patch already contains, and the -t option verifies a patch against a pair of paths without writing anything, which belongs in CI rather than in a manual pass after a failure.

What none of this carries is filesystem state, and the README is blunt about it: this library does not deal with file metadata, such as file last write time, permissions, link file, and a file is just a stream of bytes to it. Symlinks, permission bits and executable flags have to be re-established by whatever installs the patched tree. For a read-only root filesystem on the device, that is a design constraint to settle before the first rollout, not a surprise to debug during an update.

## The Makefile refuses to build rather than guessing

Every compression and format choice in this project is a variable at the top of the Makefile, and several combinations are guarded so an inconsistent request stops the build with a message instead of producing a binary with a silent hole. bsdiff support needs bzip2, and libdeflate cannot be linked against a system zlib, so asking for both is an error rather than a fallback:

```makefile
$(error error: support bsdiff need BZIP2! set BSD=0 or BZIP2>0 continue)
```

That guard sits above a second one covering libdeflate with zlib, and a third case only warns: with libdeflate enabled and no zlib source compiled in, the build continues even though libdeflate cannot handle every deflate stream zlib can produce.

The same block decides whether directory diffing, multi-threaded patching, VCDIFF and bsdiff support are compiled at all, whether the output is 32-bit or stripped for size (M32, MINS), and whether arm64 assembly and a software CRC are used for lzma. It also carries a stray comment: the line `# supported atomic uint64?` sits directly above the comment describing the zstd switch, matching neither the variable below it nor the one it was written for. Harmless to the compiler, but a fair sign that this switch list was assembled over years.

## Android and MCU patching live outside the core library

Small targets are served by companion projects rather than by a flag in this one. For embedded systems, MCUs and NB-IoT hardware, the HPatchLite demo paired with the tinyuz decompressor is described as running on 1KB RAM devices, and it also carries an inplace patch implementation aimed at storage-constrained flash.

Android is split three ways. ApkDiffPatch handles Apk, Jar and Zip files and produces smaller deltas, yet it cannot be used by an Android app store, because diffing requires re-signing the Apk first. sfpatcher exists for exactly that gap, works in the manner of Google's archive-patcher without re-signing, and is described as running with O(1) memory.

For an in-process Java patch, the NDK route produces libhpatchz.so plus a wrapper class:

```bash
cd HDiffPatch/builds/android_ndk_jni_mk
build_libs.sh
```

The build produces *.so files there, and com/github/sisong/HPatch.java sits under java/ in the same tree. Import both and Java code can call the patch function in libhpatchz.so directly, which is the route to take when the file being patched never appears at a filesystem path your code can name.

## No old file to compare against: hsynz and hsign_diff

The working assumption behind hdiffz is that you hold both the old file and the new one when you diff. When that fails, the same author keeps separate repositories for the alternatives.

hsynz works like zsync: the new version is published once, and the owner of an older copy fetches just enough information to diff and patch locally, so the delta never travels. It adds zstd compression, is described as running much faster than zsync, and keeps compatibility with zsync's file format. That fits a catalogue of older installs with no spare copy of the previous version to hand.

A second gap appears when you hold the new file but not the old one, and can obtain a hash certificate file (.hsyni) for the old file. In that situation the hsign_diff command in hsynz produces an hpatchz format patch from the new file plus that certificate, which the README frames as the rsync scenario. Both tools sit outside this repository, so adopting either means tracking and building a second codebase, and nothing here documents how their formats relate to a plain HDiffPatch patch beyond the shared name.

## Where bsdiff4 and xdelta3 patches still apply

The alternative question is not which library is smaller but whose patches your consumers already hold. A team that standardised on bsdiff4 or endsley/bsdiff can keep those producers and still patch with this library, since hdiffz_import_patch.c sits in the tree and the readers live in bsdiff_wrapper/ and vcdiff_wrapper/. A team on xdelta3 or open-vcdiff sits in the same position, because compatibility with the VCDIFF patch file format from RFC 3284 is documented.

The real difference shows up on the producing side. In the default -m mode, this library states that its goal is the best diffFileSize, and pays for that by loading both files into memory; the streaming mode gives up some of that gain to keep the footprint flat. zsync and rsync are not patch producers in that sense at all: zsync transfers only the blocks a receiver is missing, and rsync reconciles against signatures, so neither needs a delta file shipped by the publisher, and both need the source at update time.

That narrows the choice. If you publish a delta alongside a product, this library's own format with the streaming mode is the shortest path. If devices pull updates over HTTP, hsynz or plain zsync keeps the delta off the wire, and everything here becomes the wrong tool. The badge in the README says MIT while GitHub's licence field for the repository reads Other, a custom licence the classifier cannot place, so anyone embedding this in a shipped product has to open the LICENSE file rather than trust the badge.

## Conclusion

HDiffPatch earns a place in any pipeline that ships a binary delta to a device it cannot physically touch, and its streaming mode puts a ceiling on memory instead of multiplying it by five. It is the wrong choice for signed Android packages, for trees where permissions or symlinks carry meaning, and for pull-based updates where hsynz and zsync keep the delta off the wire. Before you commit, build from the clone URL matching your intended version, then generate one patch with `hdiffz oldPath newPath outDiffFile -WD -s-64` and put it back with `hpatchz oldPath diffFile outNewPath` to confirm the round trip yourself.

## FAQ

### How do I create and apply a delta with HDiffPatch from the command line?

Write the patch with `hdiffz oldPath newPath outDiffFile -WD -s-64`, then apply it with `hpatchz oldPath diffFile outNewPath`. The -s-64 form matches in 64 byte blocks as a stream rather than loading both files into memory.

### Which HDiffPatch diff mode uses less memory, -m or -s?

The -m mode needs the new file size plus five times the old file size, or nine times once the old file reaches 2GB. The -s mode works from streams and needs roughly oldFileSize*16/matchBlockSize plus matchBlockSize*5 per parallel thread, with 64 bytes as the default block size.

### Can HDiffPatch build an APK patch that the Android app store accepts?

No. ApkDiffPatch cannot be used by an Android app store, because diffing an Apk requires re-signing it first. The author points at sfpatcher for that case, which works like archive-patcher without re-signing and runs with O(1) memory.

### Does HDiffPatch keep file permissions, timestamps and symlinks when patching a directory?

It does not. The library does not deal with file metadata such as file last write time, permissions or link files, and treats a file as a stream of bytes. Whatever depends on that metadata has to be restored by the installer or by extending the library.

### Can an HDiffPatch patch be applied on a microcontroller with 1KB of RAM?

The README points to the HPatchLite demo, which with the tinyuz decompressor is described as running on 1KB RAM devices, and which also offers an inplace patch implementation for storage-constrained devices.

## Sources

- [Issues](https://github.com/sisong/HDiffPatch/issues)
- [README](https://github.com/sisong/HDiffPatch/blob/master/README.md)
- [Releases](https://github.com/sisong/HDiffPatch/releases)
- [sisong/HDiffPatch on GitHub](https://github.com/sisong/HDiffPatch)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/sisong-hdiffpatch