Mergen, lifting obfuscated x86 functions into LLVM IR

It lifts a function, not an instruction

Mergen is a tool engineered to convert assembly code into LLVM Intermediate Representation, and the README is precise about the unit of work. The project symbolically executes, or symbolically lifts, the target, and the idea is not lifting individual instructions but lifting a whole function.

That distinction explains the design. The authors do not expect one instruction, or one basic block, to behave the same each time it runs; instead they treat them as things that can be, and are, for different purposes each time. A traditional lifter maps instructions one to one and inherits the obfuscation. Mergen tries to recover the meaning, so the generated IR stays simple and optimizable.

The stated objectives are deobfuscation, devirtualization and optimization, and the purpose is framed as making the reverse engineering process more efficient and effective for complex software systems. The repository topics match that framing: deobfuscation, devirtualization, llvm and optimization.

The tooling around it is what makes this a serious reverse engineering project rather than a script. The tree includes an ARCHITECTURE.md, LLVM_API_NOTES.md, a REVIEW_CHECKLIST.md, VMP_TESTING_NOTES.md, an autoresearch.md with an autoresearch.sh script alongside it, and a testcases/ directory holding the assembly inputs the examples are built from. The lifter/ directory holds the tool itself, linux-pe carries PE handling, and a Zydis-4.1.0/ directory provides the disassembler. There is also an icpped_rust/ directory, and the presence of an AGENTS.md at the root says the project expects assistant-driven work on it.

The licence is GPL-3.0, the newest release tag is v0.7 from 2026-03-29, and the last push to the repository was on 2026-09-22.

It does not trust LLVM's analyses for its own job

The most revealing sentence in the README explains why the tool cannot simply lean on the compiler it targets. Mergen uses analysis to evaluate control flow, and it cannot depend on LLVM for all of its analysis, because LLVM's analyses are created for different goals and could be unoptimal for this use case.

That is a real design position rather than a complaint about LLVM. An optimiser's job is to make fast code faster, and LLVM's analyses are built with that objective in mind. A lifter's job is to make obfuscated code readable again, which sometimes means being deliberately conservative in a way an optimiser would not be, and sometimes means understanding a program's control flow in a way that has nothing to do with hot paths or register pressure.

So Mergen does its own analysis and emits IR that LLVM's optimiser can then work on. The division of labour is that Mergen decides what the function means, and LLVM's optimiser decides how to arrange the instructions once the meaning is clear. That is also why the examples in the README show both an unoptimized output and an optimized one, and why the project cares about the IR staying optimizable in the first place.

The practical consequence for a user is that the output is ordinary LLVM IR. You can compile it, feed it to a decompiler, or hand it to the optimiser, and the result is a text file you can read and diff, which is the thing an obfuscated binary does not give you.

The VMProtect example shows the whole workflow

The first worked example is the clearest picture of how the tool is used, and it starts with a deliberately trivial target:

struct test {
    int a;
    int b;
    int c;
};

int maths(test a, int b, int c) {
        return a.a  + b - c;
}

The function is then protected with VMProtect, with the settings described as everything turned off and the function virtualized on the ultra setting, and the tested VMProtect versions listed as 3.4.0 to 3.6.0 and 3.8.1. The README shows both the original disassembly and the original decompilation, so the difficulty of the protected function is visible before Mergen touches it.

Running the tool is described as simple. The first argument is the name of the file and the second is the address of the function. The output is an LLVM IR module, and the README notes that you can compile the output so you can explore it using your favourite decompiler.

The generated IR for the example shows what the lift buys. The virtualized arithmetic becomes a handful of ordinary operations on a 32-bit value loaded from memory, with the flags computation gone, and the function returns the result:

entry:
  %stackmemory = alloca i128, i128 13758960, align 8
  %1 = trunc i64 %r8 to i32
  %2 = trunc i64 %rdx to i32
  %GEPLoadxd-5369456437- = getelementptr i8, ptr %memory, i64 %rcx
  %3 = load i32, ptr %GEPLoadxd-5369456437-, align 4
  %adc-temp-5370242400- = sub i32 %2, %1
  %realnot-5369532059- = add i32 %adc-temp-5370242400-, %3
  %stackmemory10243.sroa.55.1375304.insert.ext10255 = zext i32 %realnot-5369532059- to i64
  ret i64 %stackmemory10243.sroa.55.1375304.insert.ext10255

The obfuscated control flow is gone. What is left is the original arithmetic, and it can be compiled and decompiled like any other IR.

Indirect jumps collapse because values are tracked

The second example is about the case that frustrates reverse engineers most often, the indirect jump. The README gives the plain source that a virtual machine would transform into an indirect jump, and calls it slightly more inconvenient for the reverser.

The tool's answer is that it always analyses values and keeps track of them, which is what allows it to understand control flow. The test input lives in the repository at testcases/test_branches.asm, and the README shows both the optimized and the unoptimized output.

The difference between them is the point. The unoptimized IR still contains the full flag computation, with the multiply and mask sequence used to test a bit, the zero and sign flag derivations, the stores of the reconstructed flags into memory, and the branch on the computed condition to two return blocks. That is a faithful lifting of the machine behaviour.

The optimized output reduces the same thing to a shift, an and, and a return. The logic has been recovered rather than merely translated.

This is the same principle as the VMProtect case applied to control flow instead of arithmetic: keep the meaning, discard the mechanism. A virtual machine's job is to make one operation look like many, and a lifter that tracks values long enough to know what the dispatcher was really computing undoes that.

The registers are wrong, and that is expected

One detail in the README is worth quoting because it will otherwise look like a bug. After the VMProtect example, it notes that the registers are a little bit off, and explains why: the tool does not follow the calling conventions, and if it did, the function signature would look different.

The signature the tool emits takes the machine registers in a fixed order, with a pointer to memory at the end, rather than the order the Windows x64 convention would use. The README's response is to adjust the function signature to look normal, and to point the reader at the calling convention and ABI documentation for the parameter passing rules and register volatility.

So the output is not directly compilable as a normal C function without that manual step. The IR is correct about what the machine did, and it is simply not dressed as an ABI-conforming call.

The practical advice is to treat the lifted function as a description of behaviour rather than as a drop-in replacement for the original, and to spend the small amount of time needed to fix the signature before handing it to a decompiler. Anyone expecting a clean compilable module from a one-command invocation will be disappointed, and the README is upfront about that.

The rewrite baseline gate is the regression safety net

Because the output is IR that is compared and reviewed, the project needs a way to notice when its own rewriting breaks, and it has one. The README describes a rewrite baseline gate that rewrite work should keep green. The gate builds focused PE samples, runs lifter, and verifies the lifted IR outputs, with a workflow document at docs/REWRITE_BASELINE.md and a one-command gate at scripts\rewrite\run.cmd.

The existence of that gate is the most useful signal in the repository about how the project is worked. A lifter is exactly the kind of tool where a change can silently alter the meaning of a lifted function while still producing valid IR, so a test that compares lifted output against a stored expectation is the minimum viable protection. The README also carries VMP_TESTING_NOTES.md and a testcases/ directory, and the repository has a test.py at the root.

The build documentation is separate, in docs/BUILDING.md, and the Dockerfile shows what the build needs in practice. The image is based on ubuntu:22.04, installs the LLVM 18 toolchain from apt.llvm.org, optionally installs Rust through rustup when the INSTALL_RUST argument is true, symlinks clang-18 as clang and clang-cpp-18 as both clang-cpp and clang++, and then configures and builds with CMake:

cmake .. && cmake --build . -j $(nproc)

The container's default command copies the built binary to /output/lifter, which tells you the binary name to expect.

Editorial conclusion

Mergen suits a reverse engineer who has hit a virtualised or control-flow-flattened function and wants a readable, optimisable IR version of it rather than another hand-traced state machine. It does not suit automated bulk analysis or anything outside the x86 world it targets, and the produced IR is not calling-convention clean. Before you rely on it, build it against LLVM 18 as the Dockerfile does, run the rewrite baseline gate with scripts\rewrite\run.cmd so you know your tree still reproduces the expected lifted output, and treat every lifted function as needing manual signature fixing before it compiles.

Frequently asked questions

Official sources

1. Issues
2. License: GPL-3.0
3. NaC-L/Mergen on GitHub
4. README
5. Releases