gateGPT: a microGPT transformer in Verilog on a Virtex-5 FPGA
Full Transformer into a custom chip. microGPT in RTL, generating names on a Virtex-5 FPGA at ~56k tokens/second.
At a glance
- What is it?
- gateGPT puts a one-block character-level GPT into RTL and runs it on a Xilinx Virtex-5 board, generating names on an LCD at roughly 50,000 to 69,000 tokens per second. It is a reference design for hardware people, not a drop-in inference stack.
- Who is it for?
- Adopt gateGPT if you want a small, fully documented case study of transformer inference in RTL: fixed-point spec, microcode ISA, actuators, and a Python reference that is bit-exact to the hardware. Do not adopt it as a general LLM accelerator; the model is one block with n_embed=24, context 16 and a 27-token vocabulary, and the repository does not document a licence.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 84 days ago.
- What is it written in?
- Mainly Verilog, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 17, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What gateGPT actually is, and who should care
gateGPT is a hardware implementation of Andrej Karpathy's microGPT, a small character-level GPT, written in Verilog-2001 and targeted at a Xilinx Virtex-5 FPGA (XC5VLX110T on the XUPV5 / ML509 board, built with ISE 14.7). The model is one transformer block: RMSNorm, then multi-head causal attention, then an MLP, with n_embed=24, 4 heads of dimension 6, MLP hidden size 96, context length 16 and a vocabulary of 27 characters (a dot plus a through z). Every arithmetic operation is signed Q5.11 fixed point. The board generates names on its character LCD, and a rotary encoder changes generation speed and sampling temperature.
The audience is narrow and specific. This is for engineers who want to see a transformer schedule expressed as RTL: what a matvec tile looks like, how RMSNorm is built from an integer square root and a reciprocal, how a softmax divide is scheduled across heads. It is not for someone who wants to run a language model in an application. The README is explicit that the RTL, the fixed-point spec, the microcode ISA and the trained weights are all the author's own, so the value here is the design and its verification story, not a reusable accelerator IP block.
Microcode sequencer, datapath actuators, and a shared scratchpad
The inference core is not a monolithic state machine. It is a microcode-ROM sequencer driving modular datapath actuators. A small program ROM, generated/ucode.hex, is produced by tools/ucode_asm.py and encodes the transformer schedule as macro-ops. A micro-PC fetches one macro-op per step, starts the matching actuator, and waits for its done signal. Only one actuator is active at a time.
The actuators live in core/: matvec for the linear projections (a 24-lane, 2-columns-per-cycle MAC tile), norm for RMSNorm, attn for single-position multi-head causal attention with per-head parallel dividers, exp_unit for fixed-point exp via a 17-entry table plus linear interpolation, sampler for temperature softmax and LCG categorical sampling or greedy argmax, plus embed and vecop for embedding lookup and residual add or ReLU. Weights, RMSNorm gains and the microcode come from separate ROMs (wrom, grom, ucode), and the working set plus the persistent KV cache share one true dual-port Block RAM called vmem.
The data flow is what makes the throughput numbers plausible. Incremental decoding with a persistent KV cache means each step computes only the new token's K and V and attends over the cached context, instead of recomputing the whole 16-token window. The README's optimization table shows that change alone moved the design from 32,872 cycles per token to 10,192. The 24-lane matvec tile then took it to 2,757, and later stages (parallel attention dividers, a radix-4 divider, a narrow isqrt, dual-port vmem, two columns per cycle in matvec) brought it to 1,156 cycles per token in the final design.
Building the bitstream and watching a name appear
The repository ships two Tcl scripts at the top level, build_board_ise_project.tcl and run_board_bitgen.tcl, for the ISE 14.7 flow, plus board/, core/, sim/, tools/, generated/ and data/ directories. The README does not spell out a step-by-step install sequence, so the honest first step is to read the scripts themselves and the board/ directory before running anything. The microcode ROM is not checked in as source; it is generated by tools/ucode_asm.py, which the README names as the producer of generated/ucode.hex. The README gives the script names but not the exact invocation, so the anchor points are those two files:
tools/ucode_asm.py
build_board_ise_project.tcl
run_board_bitgen.tclOnce the bitstream is on the XC5VLX110T, the README states the design generates names on the board's character LCD, with the rotary encoder setting generation speed and sampling temperature. The reported throughput at 80 MHz is about 69,200 tokens per second for the first token, about 60,600 averaged over a full name, and about 53,800 for the longest-context token. Those are the author's post-place-and-route figures, not measurements reproduced here.
DSP48E at 96 percent is the wall you hit first
The resource table in the README is the most useful limitation in the whole document. On the XC5VLX110T-1 FF1136, the final design uses 16,548 slice LUTs (23 percent), 5,530 slice registers (8 percent), 2 Block RAMs (1 percent) and 62 of 64 DSP48E slices. That is 96 percent of the DSPs. The README states plainly that DSP is the binding resource, and that the 24-lane by 2-column matvec tile alone uses 48 of the 62.
That single line tells you where the design cannot go without restructuring. There is no headroom to widen the matvec tile, add a second transformer block, or grow n_embed on this part. The LUT and register numbers look comfortable, but they are not the constraint. Anyone planning to extend gateGPT should treat the DSP budget as the design's boundary, not its utilization percentages in aggregate.
The second limitation is toolchain age. ISE 14.7 targets Virtex-5, a family long past its mainstream support window, and the build depends on that specific flow. The README does not describe a Vivado port, and the repository layout gives no sign of one. If your lab no longer has an ISE 14.7 installation or an XUPV5/ML509 board, the practical path is the simulation flow in sim/ and the Python reference in tools/fixedpoint.py, not the board.
How it differs from a software inference runtime
The obvious alternative for someone who wants a small character-level GPT is to run microGPT itself in PyTorch or NumPy on a CPU. The difference in approach is total. The Python reference in tools/fixedpoint.py exists to be the bit-exact specification that the RTL matches, not to be fast; the hardware exists to execute that same integer arithmetic in a fixed pipeline. In software you get flexibility and a training loop; in gateGPT you get a fixed Q5.11 datapath, a fixed context of 16, and a fixed vocabulary of 27, with no way to change the model shape without changing the RTL and the ROM contents.
That trade is the point. The README reports that throughput improved 28 times over the first working version, from about 2.4k to roughly 50k to 69k tokens per second depending on context length, and that every optimization stage stayed bit-exact to the Python reference, verified in an iSim oracle with named greedy and sampled outputs (alaya and rosphod at seed 2, T=0.7). A software runtime would not need that verification discipline because it would not be reimplementing exp, isqrt and division in fixed point. gateGPT's alternative is not a faster CPU inference path; it is a demonstration that the arithmetic can be pinned down exactly and then scheduled in hardware.
Maintenance, licensing and what the repository does not say
The last push to the default branch was on 2026-06-25, which is under six months before today. The repository is not archived. There are no releases retrieved, so there is no versioned artifact to pin and no changelog to read. Upgrades therefore mean tracking the main branch and regenerating generated/ucode.hex with tools/ucode_asm.py whenever the microcode source changes, then rebuilding through the ISE Tcl scripts.
The licence is listed as unknown. That is a real blocker for anything beyond personal study: without a licence file, the default position is that the author retains all rights, and redistributing the RTL, the weights or a bitstream is not something the repository grants. This is not legal advice, but it is a concrete reason to contact the author before using gateGPT in a product or a published derivative. The README also does not document rollback, versioning or a supported upgrade path, so treat the main branch as the only source of truth and expect to re-verify bit-exactness after any pull.
Editorial conclusion
Adopt gateGPT if you want a small, fully documented case study of transformer inference in RTL: fixed-point spec, microcode ISA, actuators, and a Python reference that is bit-exact to the hardware. Do not adopt it as a general LLM accelerator; the model is one block with n_embed=24, context 16 and a 27-token vocabulary, and the repository does not document a licence. Before building anything on it, check the licence field, confirm the ISE 14.7 toolchain and XUPV5/ML509 board are available to you, and run the Python reference in tools/fixedpoint.py against the RTL simulation in sim/ to see the bit-exactness claim for yourself.
Frequently asked questions
What is a generative pre-trained transformer (GPT), and how does gateGPT relate to it?
gateGPT is a hardware implementation of Andrej Karpathy's microGPT, a small character-level GPT, in Verilog-2001 on a Xilinx Virtex-5 FPGA. The model is one transformer block with RMSNorm, multi-head causal attention and an MLP, and it generates names on the board's character LCD.
Is an FPGA faster than a CPU for gateGPT?
The README reports about 69,200 tokens per second for the first token and about 60,600 averaged over a full name at 80 MHz, but that is for a one-block model with n_embed=24, context 16 and a 27-token vocabulary. It is not comparable to a CPU running a general language model.
Is FPGA still used, and where can I find the gateGPT source?
gateGPT targets a Xilinx Virtex-5 FPGA (XC5VLX110T, XUPV5 / ML509 board, ISE 14.7), and the project is the repository fguzman82/gateGPT. The inference core is under core/, the simulation under sim/, the Python reference and microcode assembler under tools/, and generated ROM contents under generated/.
Community notes