Handwired: turning ordinary JavaScript into a differentiable neural net

A conditional becomes arithmetic, not a branch

The core idea is stated in one line: a program is made of if-else, loops and state, and Handwired transforms them into matrix multiplication and ReGLU over floats. The example is small enough to check by hand.

Starting from an ordinary JavaScript conditional:

const a = 5
const result = (a > 4) ? 10 : 12

The rewritten version computes both branches and selects between them with arithmetic:

const a = 5
// Math.max(0, x) is ReLU. For an integer a, above4 is 1 if a > 4, and 0 otherwise
const above4 = Math.max(0, a - 4) - Math.max(0, a - 5)
// Both branches compute, and above4 picks one
const result = above4 * 10 + (1 - above4) * 12

The trick is that `Math.max(0, x - 4) - Math.max(0, x - 5)` is a ramp: zero at 4 or below, and one at 5 and above for integer inputs. Multiplying by 10 and by `1 - above4` weights the two branches, so the sum picks exactly one of them.

The claim that this is exact, not approximate, is what separates the project from a compiler that fakes control flow with masking. The documentation invites you to calculate it by hand and see that both versions arrive at the same result.

Only two primitives, which is why the GPU is happy

Once every conditional is arithmetic, the whole program reduces to a small set of operations. The project then collects the addition, the multiplication and the ReLU into a giant matmul, and either runs it on JavaScript or on WebGPU.

The reason only two primitives matter is spelled out. The full program stays differentiable, because a ReLU has a derivative almost everywhere and a multiplication differentiates into a product rule. The weights do not need training since they are written manually, which is the point of the whole exercise. And existing GPU techniques such as fusion, sparsity and reordering have an easy time optimising such minimal primitives, since there is very little variety in the operations to schedule.

That last point is the practical reason this approach is interesting on hardware. A conventional neural net has a fixed layer structure that the compiler knows in advance. A compiled program has an irregular control flow graph that the compiler has to turn into something the hardware can chew on, and reducing everything to matmul and ReLU is how you get back to the shape hardware is good at.

The project lists three directions this opens up: unfreezing the weights and training them, which the README phrases as a question, attaching a specific frozen program to a generic neural net and training end to end, and achieving GPU level parallelism with JavaScript code that was written in a traditional single threaded way, with a JSON search demo that runs in milliseconds over 100k documents as the example.

Running the demos needs Bun, and the repository is a lab

The demo site is published live, and the local setup is short. The README gives the sequence as a clone, an install and a start:

git clone && bun install
bun start

The `start` script in the package manifest runs `bun scripts/serve-demo-site.ts`, so the dev server serves the demo site. The manifest also carries a set of `start:*` scripts that open individual pieces directly, including a candidate gallery, a hard attend experiment, a Knuth and Plass demo, a learned gallery, a reusable layout demo and the template directory.

The repository layout tells you where the value is. There is a `docs/` directory, a `src/` directory and a `template/` directory, and the README says to check those to get started, while `lab/` is described as a bunch of old experiments that you can check or ignore. That last remark matters for anyone assessing the project: the interesting-looking directories are not all the same, and part of the tree is explicitly archival.

Other infrastructure is present as well. A `tests/` directory, an `oxlint` configuration in `.oxlintrc.json`, a `knip` configuration for unused code detection, a `bun.lock` lockfile and a `tsconfig.json`. The `test` script runs `bun test` with a 30 second timeout across the tests, the template and the lab directories, so the old experiments are still exercised by the suite even though they are not where you should start.

There is also a `check` script, a `knip` script and site scripts for building and checking the demo site, which suggests the project treats its own public demos as something worth testing.

What a frozen program inside a trainable net actually does

The three directions listed in the README are worth separating, because they differ in how much of the original program is allowed to change.

The first is unfreezing the weights and training them. The weights in this system are not learned values in a layer, they are the constants written into the transformed program, the thresholds, the branch values, the literals. If you train them, you are not adjusting a model in the usual sense, you are letting the numbers in your if-else become continuous parameters. The README is honest that it does not know what this means, phrasing it as a question to find out, which is a good sign about the state of the investigation rather than a bad one.

The second is attaching a specific frozen program to a generic neural net and training end to end. The example given is a nanoGPT that could then do both Shakespeare and a particular UI. Here the program is fixed and the net around it is learned, so the frozen behaviour becomes a deterministic component inside a trainable pipeline.

The third is parallelism rather than learning at all. A JavaScript program written for one thread is compiled to a form the GPU can run across many cores, and the JSON search demo over 100k documents is the demonstration. Nothing is trained and nothing changes, the work is simply redistributed.

The distinction between the first two is the one to keep straight. One changes the program's own numbers, the other keeps the program and trains something attached to it, and they have very different failure modes.

Where the project stands and what it is not

Handwired is presented as something to play with rather than as a product with a support commitment. There are no GitHub releases, the package in the manifest is marked private, and the project is described by its own README as a place to try the demos and look at the source.

The last push was on 2026-10-17 and the repository is not archived, so the work is current, and the licence is MIT.

The technical limits are inherent rather than accidental. WebGPU is required for the GPU path, and the README does not discuss a fallback or a compatibility matrix, so browsers without WebGPU or platforms without a GPU backend are outside what is described here. The JavaScript path exists as an alternative, and the project says it runs the matmul on JS or on WebGPU, for fun.

The other limit is exactness of scope. The rewrite works because a program can be expressed with addition, multiplication and ReLU, and the demonstration covers conditionals. Programs that depend on side effects, on floating point ordering, on integer overflow behaviour or on the exact semantics of a comparison are the interesting edge cases, and the README does not discuss them.

The alternative approach is a conventional differentiable programming system, where you write the program in a language with a compiler that emits differentiable operations, and you accept that the operation set is larger and the semantics are defined by that language rather than by arithmetic rewriting. The difference in approach is that Handwired takes code you already have and changes its representation, while a differentiable language asks you to write the program in a dialect from the start. Handwired is the more interesting experiment and the more fragile foundation, since the correctness argument rests on a transformation you have to trust.

Editorial conclusion

Handwired is worth an afternoon for anyone who wants to see what a frozen, hand written program looks like inside a trainable network, because the transformation is exact rather than an approximation, and the repository ships a set of demos including a JSON search over 100k documents. Do not adopt it as a production dependency, since it is described as a project to play with, has no GitHub releases, and the interesting behaviour lives in the `lab/` directory of old experiments. Before you build on it, check that the browsers or runtime you target support the WebGPU path, and read `docs/` and `src/` for the current structure rather than `lab/`, and note that the last push was on 2026-10-17 with the package marked private and unpublished.

Frequently asked questions

Official sources

1. chenglou/handwired on GitHub
2. Issues
3. License: MIT
4. Project website
5. README