apex-inference-chip: one decoder layer, bit-exact, and honest about not being silicon
An inference chip design that runs a real LLM (Qwen2.5-0.5B) on FPGA — one transformer decoder layer in RTL, every silicon value bit-exact against a golden model. 0.56 tok/s measured, a 140× climb, full evidence trail.
At a glance
- What is it?
- An open RTL design for a single transformer decoder layer whose blocks are checked bit-for-bit against an executable NumPy reference, with a KV-cache codec inside the datapath. It ran a half-billion-parameter model on FPGA, and every number in the repository is labelled measured or projected.
- Who is it for?
- The verification discipline is the reason to look at this, not the throughput: bit-exact equality against an executable reference, mutation-tested testbenches and machine-generated evidence under a stated anti-fabrication rule are a stronger claim than a benchmark curve, and the bit-exact KV codec is a genuinely interesting place to put the complexity. Two limits to hold on to.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 55 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 12, 2026, and from our analysis. They are not legal advice.
Editorial analysis
Every number is labelled measured or projected, and the page says why
The opening paragraph sets the rule before making any claim. This is a verification-first, fully open tile, and the project describes itself as pre-silicon with every number in the repository labelled as what it is, measured or projected. That is a stronger opening than most hardware projects manage, because the alternative is a performance table where projections and measurements sit in the same column and the reader has to guess. The repository structure supports the claim: a status file, a results index, a master design table, a traceability document, and separate directories for the hardware description, the golden model, the reference implementation, the evaluation harness, the performance work and the verification environment. When a number in this project is projected, you can follow the pointer rather than take it on faith, and when one is measured, the same trail applies.
Bit-exact means bit-exact, and the testbenches are mutation-tested
The methodological bet is stated as plainly as the architectural one. Nothing ships without a bit-exact golden model: every hardware block in the tree is checked against an executable NumPy reference, and the page is explicit that the standard is not close and not within tolerance but bit-identical. Two supporting mechanisms make that credible rather than aspirational. The testbenches are mutation-tested, which is the practice of deliberately breaking the design to confirm the tests notice, so a passing suite means something specific. And the evidence is machine-generated under an anti-fabrication rule, which is an admission that hand-written evidence logs are worthless in a hardware project. Neither of those is unusual in a serious verification shop, and both are unusual in a public repository, which is why the difference between this and a hobby core is worth noticing.
One matrix engine, time-multiplexed across seven jobs
The tile's central resource is a single INT8 systolic array, described as the MXE, and it is time-multiplexed across every matrix operation a decoder layer needs: the three query, key and value projections, the two attention products, the output projection, and the feed-forward matrices. A sequencer choreographs them. That is seven jobs on one engine, and it is the design decision that keeps the tile small enough to be described as one tile. The second idea is that the KV cache never lands in SRAM as full-precision data: keys and values are quantized in flight by a dedicated codec engine and dequantized in flight on the read side of the attention products. The third is an online softmax that streams scores, so there is no materialised score matrix and no buffer whose size grows with the context length.
A hardware unit decides which context is worth its bits
The adaptive-precision story has a dedicated block, and it is the most unusual idea in the design. A unit tracks which cached tokens keep mattering and drives the codec's precision tier per region of the cache, choosing between a byte-wide tier, a four-bit tier, and a four-bit tier with outliers kept separately. So the bit budget is not uniform across the conversation: recent or important context gets more precision, stale or repetitive context gets less, and a small outlier lane catches the values that a four-bit codec would flatten. The page is careful to attribute the adaptive part to that tracking unit rather than to the quantiser, which is the right way to draw it, since the novelty is in deciding per region rather than in the quantiser itself.
Scales travel with the data, so a job is reproducible from its descriptor
Two properties tie the blocks together into a machine rather than a pile of blocks. The first is that quantization scales for activations and composites move through the same seams as the tensors they describe, which means a job is bit-exactly reproducible from its descriptor alone, with no hidden state on the other side of a boundary. The second is that every hardware operation is a self-describing descriptor carrying matrix shapes, routing and the quantization contract. That second property is what the page credits for making host-driven execution and walker-driven execution the same machine, and for making every flight replayable in simulation bit for bit. It is also what lets the project ship a software runner alongside the hardware, which is the practical way to test anything without a board plugged in.
What the scope excludes is stated in the same breath as what it proves
The honest-scope paragraph is unusually direct. This is one tile, with no memory controller, no peripheral bus and no network-on-chip, all of which are declared out of scope by charter rather than accidentally missing. The architecture is demonstrated end to end on a half-billion-parameter Qwen model on FPGA hardware, and it is sized for the 7B class, where a head dimension of 128 exists in the hardware description. Then the sentence that matters for anyone quoting the larger model: those tokens have run through the software-verified golden pipeline, and explicitly not through silicon. The summary in the repository metadata adds the measured throughput of 0.56 tokens per second and calls it a hundred-and-forty-fold climb from wherever the earlier number was. So the demonstration is real, the 7B story is a design target, and neither is a product.
A vendor section sits inside the technical documentation
Partway through the architecture chapter the page becomes a product page for the company that built it. The project was developed with the vendor's verification tooling, which is described as autonomous verification agents that take hardware from a natural-language specification to verified hardware description, with real testbenches, real simulators and honest sign-off. Underneath that is a named planner engine described as treating verification as scientific discovery, choosing at every step the one simulation that will reveal the most about the design. None of that is verifiable from this repository, unlike the hardware claims. That is worth stating plainly rather than treating as a fault: the technical material is unusually checkable, the tooling material is a vendor statement, and a reader should weight them differently. The repository lives under the vendor's organisation and its homepage is the vendor's site.
The primary language is Python in a hardware repository, and that is not an accident
The recorded primary language for this repository is Python, which looks odd until you look at the layout. Alongside the hardware description directories sit a golden model directory, a reference implementation, an evaluation harness, performance work, scripts, and a runner at the root that executes the design in software. The verification story is executable, which means the reference model and the testbenches that drive it are code, and that code is Python. The one demo example at the root level is also Python. So the language reflects where the project's claims are checkable: the golden model is the thing everything is compared against, and it is a NumPy program you can read, run and break.
Editorial conclusion
The verification discipline is the reason to look at this, not the throughput: bit-exact equality against an executable reference, mutation-tested testbenches and machine-generated evidence under a stated anti-fabrication rule are a stronger claim than a benchmark curve, and the bit-exact KV codec is a genuinely interesting place to put the complexity. Two limits to hold on to. FPGA bring-up is not a taped-out chip, and the page says so. And the 7B-class claims run through the software reference, not through silicon, which the repository is careful to state rather than let you infer.
Frequently asked questions
What is the apex-inference-chip project?
An open, verification-first design for one transformer decoder layer as real hardware description, covering attention, KV-cache compression, softmax, normalisation, rotary position embeddings, the gated feed-forward unit and the residual. It has been brought up on FPGA with a half-billion-parameter Qwen model, and the project describes itself as pre-silicon.
How does APEX verify its hardware blocks?
Every block is checked against an executable NumPy golden model to bit-exact equality rather than a tolerance, with mutation-tested testbenches to prove the tests can fail and machine-generated evidence under an anti-fabrication rule. The repository carries a status file, a results index and a traceability document alongside the model and the verification code.
How does APEX handle the KV cache?
The codec sits inside the datapath. Keys and values are quantized in flight to per-channel four-bit keys and per-token four-bit values with a half-precision outlier lane, and dequantized in flight on the read side. A separate tracking unit decides which cached tokens still matter and drives a precision tier per region of the cache.
Has APEX run a 7B-class model on hardware?
No. The demonstrated model on FPGA is Qwen2.5-0.5B. The design is sized for the 7B class and those tokens have run through the software-verified golden pipeline, and the page is explicit that this was not through silicon.
Can I run the APEX design without FPGA hardware?
The repository ships a software runner at its root alongside the golden model, reference implementation and verification directories. Because every hardware operation is a self-describing descriptor and quantization scales travel with the data they describe, a job is reproducible bit for bit in simulation.
What is not part of the APEX tile?
A memory controller, the peripheral bus and the network-on-chip are declared out of scope by charter, so the repository covers one decoder layer rather than a complete chip. The architecture that would wrap the tile into a full chip is specified separately with per-number provenance in the design documents.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/sigmanticai-apex-inference-chip)