CodeTracer: a time-travel debugger that answers where did this value come from

The core idea: record once, then ask where the value came from

CodeTracer is a user-friendly time-traveling debugger designed to support a wide range of programming languages. It records the execution of a program into a sharable self-contained trace file, and you load the produced trace files in a GUI environment that allows you to move forward and backward through the execution and to examine the history of all memory locations.

The framing the project uses is worth repeating because it explains the design. Compared to traditional debuggers, CodeTracer gives you two major superpowers. The first is that once you capture a bug in a recording, consider it squashed, because bugs that are hard to reproduce are painful to fix and once such a bug is captured you will rarely need more than 30 minutes to track it down.

The second is that most bugs are easily revealed when you know the origin of any value in the program. The worked example is the one every developer recognises. A program prints unexpected output to a log file midway through its execution. Clicking on that specific output event in CodeTracer takes you to the precise moment and code line where it was generated. The unexpected value must be originating from some variable passed to the logging function, and with CodeTracer you can ask where did this value come from. It will find another moment in the execution, potentially multiple seconds earlier, in a completely different part of the program where that particular memory location was last written to.

Then you repeat the question one level further. The origin might be memory corruption or a genuine logical error, and either way CodeTracer reports the origin. If you end up in the correct function that computes the problematic value but another input there leads to the issue, you keep jumping backwards until you reach the root cause.

The output events are the entry points into all of this. The recording captures a user-extensible set of output events relevant to the program, and the GUI displays these events in a searchable chronological event log. So the path into a bug is usually an output, a log line, a rendered frame or a response, rather than a memory address.

The primary language is Nim, the default branch is stable rather than main, the homepage is codetracer.com, and the licence is AGPL-3.0.

Omniscience and tracepoints, the two features that are not just rewind

Two features set this apart from a traditional debugger, and both go beyond the idea of moving backwards.

The first is called omniscience. When CodeTracer breaks at a certain line of code, it knows not only the past but also the future. You can immediately see the values of all variables in the current function right next to the source code, including variables in loops where you can scroll through the iterations. Taken code branches are highlighted while the non-executed code is immediately grayed out.

That combination is the useful part. Knowing the future means you can look at a loop variable and scroll through its iterations without stepping, and greyed out non-executed branches mean the source view shows which side of a conditional actually ran, which in a debugging session is often the answer to the question you were about to ask.

The second is tracepoints. The project makes the observation that many developers transition from interactive debugging early in their careers to mostly relying on carefully placed print statements as they face harder problems in distributed and real-time systems. A tracepoint is the middle ground: it lets you see the effects of adding additional code to your program without recompiling and rerecording it. The added code can feature if statements, additional function calls and various ways to print or plot the captured data, and the output is typically produced in seconds.

That is the feature that changes the workflow. In a traditional debugger, adding a print statement means a rebuild and a rerun, and in a system where the bug takes twenty minutes to reproduce, that loop is punishing. With a recording, the instrumentation is applied to the already-captured execution, so the cost of a new question about the data is seconds rather than a full reproduction cycle.

The project's framing of the whole thing is that once you have time travel you can re-imagine how a debugger works from the ground up, and that is a fair description of what these two features do. A traditional debugger is a forward-only instrument that you drive one step at a time. CodeTracer is a corpus you interrogate, and the omniscience view and the tracepoint evaluation are both ways of asking questions of that corpus rather than of the live process.

The rest of the mechanics support the same idea. Every time you jump to a new moment in the execution, you can explore your surroundings by stepping forward or backwards, with a back button that can always get you to any previous point of interest, and the UI shows details about the past and future program states at a glance along with your position in the recorded call trace.

The open trace format, and the community recorders

The strategic question for CodeTracer is which languages it covers, and the answer is visible in the repository structure.

The initial release supports the Noir programming language, developed in collaboration with the Blocksense team, and it currently requires the use of the Blocksense Noir Compiler, which is included in the CodeTracer distribution. Noir is the smart contract language used for zero-knowledge circuits, and the topic tags confirm the direction with arbitrum, noir, c, cpp, rust, nim and the debugging and tracing tags alongside linux, macos and windows.

The stated plan is that during 2025 CodeTracer would evolve into a comprehensive Web3 development toolkit by gaining support for many additional smart contract and zero-knowledge programming languages, with most of its functionality packaged into reusable libraries that would power the creation of block explorers with advanced transaction tracing capabilities and omniscient debugging plugins for Visual Studio Code and other IDEs. The open source development is made possible by support from Aztec Network, Arbitrum Foundation and Aptos Foundation.

The mechanism that makes that extensible is the open trace format. CodeTracer uses an open format for its trace files, hosted in a separate runtime_tracing repository, and community-driven projects have started to add support for other programming languages: a Ruby recorder and a Python recorder, both under the metacraft-labs organisation.

That is the right architecture for a tracing tool. A recorder for a language produces the trace, the trace format is the contract, and the GUI is language agnostic. It also means the language coverage is a community problem rather than something the core team has to implement one language at a time.

Metacraft Labs is also developing an alternative back-end capable of working with rr recordings, which would make CodeTracer suitable for debugging large-scale programs in system programming languages such as C, C++, Rust, Nim, D, Zig, Go, Fortran and FreePascal. That is a large claim and it is stated as work in progress, so the practical reading is that the system language support is behind an alternative back end rather than in the shipped default path.

The default branch being named stable is worth noting for anyone tracking it, and the repository root has a ct-dap.md document, which suggests an implementation of the Debug Adapter Protocol, the interface that lets a debugger integrate with an editor such as VS Code.

The build: Nix, just, and a Python recorder plan

The build system is unusual and it tells you how the project is developed.

There is a justfile at the top level, and the recipes are small wrappers around shell scripts. build runs bash scripts/build.sh, and build-once runs bash scripts/build-once.sh, and the rest of the file contains consistency checks rather than build steps. install-portable-git-hooks installs the commit and push checks that the Nix dev shell declares, and it exists for native Windows, which does not have the Nix shell, and env.ps1 calls it. A portable-pre-commit-doctor recipe runs a Python script that reports, hook by hook, which of the checks the host can actually run and the command that installs whatever is missing, and the point of it is that a missing tool fails the commit that needs it, and the doctor says so before the commit does.

Two of the checks are worth understanding because they are about build determinism. test-build-alignment asserts that just build is just build-once plus watchers and nothing else, by executing both scripts under a PATH of recording stubs for tup, webpack, livereload, repro, runquotad, nix and uname, and comparing the resulting command traces for the same host branch, the same tup variant, the same steps in the same order. The specific assertion it adds is that no webpack invocation is ordered before the first tup or repro invocation, which is the bug behind issue 599. Another check asserts that the runquota flake pin equals the runquota-src revision that its pinned reprobuild locks, because a mismatch in either direction breaks nix develop minutes in, with an undeclared identifier inside reprobuild's own sources, and attributes the failure to the wrong repository.

That is a lot of machinery for a debugging tool, and it is machinery about a class of bug that is genuinely hard, the build that works on one machine and not another. The tools named in the stub PATH, tup and repro, are the build system and the reproducible build wrapper, and the tool named runquotad appears in the flake pin.

The Nix side is equally committed, with a flake.nix and flake.lock at the root, a nix/ directory, a .envrc for direnv, and a non-nix-build/ directory for the path that does not use Nix. The .env.example file is a template you copy to .env, and its comments explain that the .envrc loads it via dotenv_if_exists so direnv and any tool reached through direnv exec or just pick it up, and that .env is intentionally not checked in.

The .env.example content is itself a good illustration of the project's testing care. It documents a visual replay integration test that records a small GL fixture and replays it through ct_gfx_player, and notes that on a host with no 3D capable GPU, such as a server with a console graphics chip or a container without a device node, the record and replay stages need the Mesa software rasterizer configured explicitly. Setting CODETRACER_VISUAL_REPLAY_SOFTWARE_GL=1 opts in, and when it is unset the harness does not touch any of those variables, so real GPU coverage is never silently disabled. The Mesa root is derived into the driver and library paths from a single root value, so only one setting changes when Mesa is bumped in the flake.

The repository also holds a large number of dated plan documents at the root, such as 0005-python-recorder-db-backend-integration.md, 0006-ct-host-reload-resilience.md and 0007-ct-host-heartbeat-and-idle-timeout.md, along with AGENTS.md, CHANGELOG.md, SECURITY.md, SUPPORT.md and a THIRD_PARTY_NOTICES.md. The numbered plan files are a way of working that keeps the design of each change visible in the repository rather than in a tracker.

The examples, and the languages actually present

The examples directory is the honest inventory of what works today, and it is worth reading as a list rather than as documentation.

The Ruby examples are the most numerous, and they are not toy programs. There is a.rb, array.rb, condition_loops.rb, fibonacci.rb, long_loop.rb, long_loop.small, loop.rb, loop.small, loop2.small, and whole project directories in ruby-project/ and ruby/, with a ruby_exit_program/ and a ruby_if_statements/ directory. The .small files are worth guessing at, and they are most likely reduced-size variants of the same programs, which is what you would want for a fast test loop.

The Python examples are two files, fibonacci.py and python_script.py.

The C example is a single file, history.c.

The Noir examples are the most structured, which makes sense because Noir was the initial target. There are noir_test/, noir_nested_loops_test/, noir_simple_sha/, noir_test_banking/, noir_test_healthcare/, noir_test_long_loop/ and noir_test_multiply_and_divide/. The banking and healthcare names tell you what they are, which is the point of the original collaboration: the tracer was built to debug a real zero-knowledge circuit stack, and the examples are shaped like the programs that stack produces.

The mix is informative. Ruby has the deepest example set, which is consistent with the Ruby recorder being the most complete of the community ones, and it is the language with the largest body of example programs to test it against. The Python and C examples are minimal, one file each, and there are no Rust, Zig or Go examples at all, which is consistent with the alternative back end working with rr recordings being a work in progress rather than a shipped path.

So the state of the project in one sentence from the examples directory: one language with first class support, two with community recorders and a handful of programs, and the system languages waiting on the rr compatible back end.

For a user picking a language, that is the check to make. If there is a recorder for your language and examples exercising it, the tool is usable today. If your language is only named in the plan for the rr back end, then you are waiting on a piece of work in progress, and the answer to the question is in the GitHub issues rather than in the documentation.

Editorial conclusion

CodeTracer fits a team debugging a bug that is hard to reproduce and who can capture a recording of it, particularly a team working in Noir today or in one of the languages with a community recorder, and who can accept an AGPL-3.0 tool and a GUI rather than a terminal debugger. It is the wrong choice if your bug only appears in production and you cannot record it, or if you need a mature debugger with a large ecosystem today, since the general purpose language support depends on recorders still in development. Before adopting, check whether a recorder exists for your language, confirm the trace file format is the open one so your recordings stay readable, and read the CHANGELOG and the numbered implementation plan files at the repository root, which show exactly what is in flight.

Frequently asked questions

Official sources

1. License: AGPL-3.0
2. metacraft-labs/codetracer on GitHub
3. Project website
4. README
5. Releases