llm-viz: A 3D Interactive Visualization of a GPT-Style Transformer

Watching a Transformer Run Inference in Three Dimensions

llm-viz displays a GPT-style language model as a 3D interactive diagram that you can explore while the model processes an input. The project description states it shows a working implementation of a GPT-style network: the network topology used in OpenAI's GPT-2 and GPT-3. The visualization runs in a web browser and does not require Python, CUDA, or any model framework to be installed.

The educational value is specificity. Most written explanations of transformer models describe how attention works in prose and static diagrams. llm-viz makes the mechanism observable: you can see the data flow through the embedding layer, through the attention heads, through the feed-forward blocks, and out through the final projection. The README describes this as a 3D interactive model of a working implementation, which distinguishes it from diagrams that show the structure without showing computation.

The project lives in Brendan Bycroft's personal homepage repository alongside a work-in-progress CPU simulation of a RISC-V processor. The LLM visualization is the primary published project. The CPU simulation is described in the README as not yet exposed.

The Demo Network: Karpathy's minGPT Sorter

The first network displayed in llm-viz comes with working weights. The README describes it as a tiny network that sorts a small list of the letters A, B, and C. This is the demo model from Andrej Karpathy's minGPT implementation.

Using an actual trained network rather than a random one means the visualization shows real inference. The attention patterns, activations, and output probabilities reflect what a trained model actually computes rather than illustrative placeholder values. The network is small enough that the full computation fits in a browser window without requiring significant hardware.

The renderer also supports visualizing arbitrarily sized networks. The smaller gpt2 size is implemented, but the README notes that the weights are not downloaded. Downloading and bundling GPT-2 weights would add hundreds of megabytes to the project. The README does not describe a path for loading those weights locally. Users who want to visualize the GPT-2 sized network with working weights would need to supply and load them outside the bundled demo.

Running llm-viz Locally

The README gives two commands for running the project locally.

yarn
yarn dev

The first command installs dependencies. The second starts the development server. The package.json defines the dev script as `next dev -p 3002`, which means the server runs on port 3002 by default rather than the standard Next.js port of 3000.

The project uses Next.js 16.3.0, React 19.2.8, Tailwind CSS 3.3.3, TypeScript 5.2.2, and d3-color for color utilities. The package.json also lists @webgpu/types as a development dependency, indicating the renderer uses the WebGPU API for graphics.

WebGPU is a modern browser graphics API available in recent versions of Chrome and Edge with hardware GPU acceleration. If your browser does not support WebGPU or if your GPU drivers do not expose the required WebGPU capabilities, the visualization may not render. The README does not list browser compatibility requirements explicitly, but the WebGPU dependency means older browsers and some virtualized environments will not work.

What the Visualization Shows and How to Navigate It

The visualization shows the complete forward pass of the transformer: the input token embedding, the positional encoding, each transformer block (with its multi-head attention layer and feed-forward network), and the final language model head that maps the hidden states to output token probabilities.

The 3D layout places the transformer's layers along a depth axis. Each layer's components are arranged spatially so you can see which tensors feed into which operations. The README describes the renderer as supporting arbitrary-sized networks, which means the same visualization engine handles both the tiny sorter model and the GPT-2 sized model differently scaled.

The project repository contains a minGPT submodule, which is the Karpathy repository the demo model's training code comes from. The gen_test_data.py script in the repository likely generates the test weights used in the demo, though the README does not document this script's purpose explicitly.

The project also includes custom font tools (create-font-atlas.jsm, create-symbol-font.jsm, fonts/) and icons, which suggests the rendering pipeline uses custom typography rather than standard HTML text rendering, consistent with a WebGPU-based approach where text is rendered as geometry.

Limitations: Fixed Demo, Weight Gaps, and WIP Scope

llm-viz ships with one working model: the minGPT letter sorter. The README confirms this is the only network with weights bundled. The gpt2 size visualization exists in the renderer but the weights are not included because they are hundreds of megabytes. There is no documented path for a user to load their own model weights into the visualization.

The CPU simulation mentioned in the README is explicitly described as work in progress and not yet exposed. It does not appear in the published visualization at the time of writing.

The project is a personal homepage repository combined with projects, not a standalone tool with a defined API or plugin system. There is no configuration for adding custom transformer architectures beyond what the renderer already supports. Teams who want to visualize their own fine-tuned models or custom architectures will find no supported extension path in the available documentation.

The README notes the project shares JavaScript utilities between the homepage and the other projects, which means the codebase is structured for Brendan's personal deployment rather than as a reusable library. Forking and adapting it for a different network topology requires understanding the custom renderer code.

llm-viz vs. BerViz: Browser Demo vs. Python Model Inspector

BerViz is a Python library that visualizes attention patterns inside transformer models from Hugging Face. The key difference from llm-viz is the connection to real models: BerViz reads attention weights from a model you load in Python, showing attention patterns for actual inputs from your own model or any Hugging Face model. llm-viz visualizes a fixed pre-trained demo model with no Python environment required.

BerViz is the appropriate tool when you have a specific model and want to inspect how it attends to tokens in a specific input. You run it inside a Jupyter notebook, load the model with transformers, pass text, and see attention head patterns for that run. This is a debugging and interpretability workflow.

llm-viz is the appropriate tool for conceptual understanding: seeing the overall architecture in three dimensions, watching how tensors move through layers, and building a spatial mental model of the forward pass. It does not connect to your models and does not require you to have any installed.

The two tools address different questions. BerViz answers what is this model attending to. llm-viz answers how does a transformer work mechanically.

Licence, Maintenance, and Repository Structure

The repository is licensed under MIT, with an explicit exception for the personal homepage files (src/homepage/, public/me.jpeg) and third-party assets such as fonts and RISC-V test data, which retain their original rights and licences. The MIT licence applies to the LLM visualization code itself. Forking the visualization code without including the personal homepage content is permissible under MIT.

The last push to the repository was on 2026-08-10. The project has no GitHub releases. The package.json name field is llm-viz and the repository is Brendan Bycroft's personal homepage repository, which means version tracking follows the commit history rather than tagged releases.

The project uses Yarn with Plug'n'Play configuration (.yarnrc.yml, .yarn/). The eslint.config.mjs provides linting configuration using eslint-config-next and react-hooks plugins. The presence of .npmrc alongside the Yarn configuration suggests historical npm usage before migrating to Yarn.

The submodule reference to minGPT in .gitmodules means cloning the repository without `--recurse-submodules` will leave the minGPT directory empty, which may affect any local scripts that reference it.

Editorial conclusion

llm-viz is the right tool for developers, researchers, and students who want to build an intuitive understanding of how a GPT-style transformer processes input by watching a real (if tiny) network execute step by step. It is the wrong choice for anyone who needs to visualize their own model weights: the demo network uses Karpathy's minGPT example, and the GPT-2 size weights require hundreds of megabytes that are not bundled. To use it, install Node.js and Yarn, then run `yarn` followed by `yarn dev` and open the browser to port 3002.

Frequently asked questions

Official sources

1. bbycroft/llm-viz on GitHub
2. Issues
3. License: MIT
4. Project website
5. README