# circuit_sparsity: inspecting sparse circuit models from Gao et al. 2025

> This repository is the open-source release accompanying OpenAI's work on understanding neural networks through sparse circuits. It provides a small GPT implementation for running forward passes on the released checkpoints, a hook system for recording activations, and a Streamlit dashboard that renders the pruned task-specific circuits, with the model weights served from a public blob store rather than committed to the repository.

**openai/circuit_sparsity** — Open-source release accompanying Gao et al. 2025

- Repository: https://github.com/openai/circuit_sparsity
- Website: https://openai.com/index/understanding-neural-networks-through-sparse-circuits/
- Stars: 540 · Forks: 57
- Language: Python
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/openai-circuit-sparsity

## A research release that ships a viewer and a small GPT

circuit_sparsity is described as the open-source release accompanying Gao et al. 2025, and the repository is titled Circuit Sparsity Visualizer and Models. It provides two things: code for running inference, and a Streamlit dashboard for interacting with the task-specific circuits found by pruning.

The framing of the whole repository is that of a paper artefact rather than a library. The homepage is OpenAI's post on understanding neural networks through sparse circuits, and the models are named after the experiments that produced them rather than after versions. The README carries a note that it was AI-generated and lightly edited, which is worth knowing before you treat any phrasing in it as a deliberate design statement.

The audience is narrow and specialised: interpretability researchers who want to look inside the pruned circuits rather than only at their accuracy numbers, and people who want to extend the analysis. That is a small group, but it is a group that would otherwise have nothing to look at, because the models and the pruned circuit data are the interesting part of the work.

The repository is not archived and the last push was on 2026-09-04, and there are no GitHub releases, which is normal for a release that accompanies a paper. The top-level listing shows the expected shape: a circuit_sparsity/ package, a tests/ directory, a pyproject.toml, a MANIFEST.in, and two utility scripts at the root, per_token_viz_demo.py and clear_cache.py.

## Installing the package and starting the dashboard

The install is an editable pip install from the repository root:

```bash
pip install -e .
```

The project requires Python 3.10 or newer, and the dependency list in pyproject.toml is heavy in a way that reflects what the visualizer needs: streamlit, plotly, matplotlib, seaborn, tiktoken, torch, blobfile, watchdog, xxhash, natsort, idemlib, pytest and mpi4py. The presence of torch is unsurprising given the forward pass code, and mpi4py points at the distributed training side of the pipeline that the pruning sweeps came from.

Launching the dashboard is a single command from the project root:

```bash
streamlit run circuit_sparsity/viz.py
```

The app loads data from the openaipublic webpage and caches locally, which is the mechanism that keeps the dashboard usable without vendoring gigabytes of model weights into a git repository. Once it loads, you choose a model, a dataset, a pruning sweep and a node budget k from the controls in the left column. The plots are rendered with Plotly, and most elements are interactive with hover and click exploration.

The screenshot in the README shows the wte/wpe tab with node ablation deltas and activation previews, which tells you the two analysis modes the dashboard is built around: which nodes matter, and what those nodes are doing.

## A deliberately small GPT and a hook recorder for activations

The model definitions live in circuit_sparsity.inference.gpt, and the module exports two things of note. GPTConfig and GPT are described as a lightweight GPT implementation suitable for CPU or GPU inference, and load_model(model_dir, cuda=False) is a convenience loader that expects the beeg_config.json and final_model.pt pair found under models/.

The example usage, adapted from the test file, shows the intended workflow:

```python
from circuit_sparsity.inference.gpt import GPT, GPTConfig, load_model
from circuit_sparsity.inference.hook_utils import hook_recorder
from circuit_sparsity.registries import MODEL_BASE_DIR

config = GPTConfig(block_size=8, vocab_size=16, n_layer=1, n_head=1, d_model=8)
model = GPT(config)
logits, loss, _ = model(idx, targets=targets)

# to get activations
with hook_recorder() as rec:
    model(idx)

# rec is a dict that looks like {"0.attn.act_in": tensor(...), ...}

pretrained = load_model(f"{MODEL_BASE_DIR}/models/<model_name>", cuda=False)
```

The hook_recorder is the part that makes this more than a model loader. It is a context manager that captures internal activations during a forward pass, and the keys follow a naming scheme such as 0.attn.act_in, where the leading number is the layer index and the rest identifies the tensor. That naming convention is what lets the analysis code address a specific node in a specific layer rather than a flat list of tensors.

Note the scale of the toy config in the example, block_size 8, vocab_size 16, a single layer, a single head and d_model 8. That is a test fixture, not a working model, and it makes the point that this implementation exists to be small and inspectable. The real weights come from the blob store, and the small GPT is what lets you run them without a framework in the way.

The tests run with:

```bash
pytest tests/test_gpt.py
```

## Where the data lives, and why nothing is in the repository

The data layout is the part of this repository that will trip you up first, so it is worth reading carefully.

Project assets live under a public blobstore URL, openaipublic.blob.core.windows.net/circuit-sparsity, and the structure is documented as follows. Under models/<model_name>/ there is beeg_config.json, the serialized GPTConfig used to rebuild the model, and final_model.pt, the checkpoint used by load_model. Under viz/<experiment>/<model_name>/<task_name>/<sweep>/<k>/ there is viz_data.pkl, described as the primary payload loaded by viz.py, containing circuit masks, activations, samples, importances and related data, with additional per-run outputs such as masks, histograms and sample buckets stored in the same tree when the preprocessing scripts produce them. Under train_curves/<model_name>/progress.json there are training metrics consumed by the dashboard's summary table. Other experiment-specific directories, for example csp_yolo1/ and csp_yolo2/, hold raw artefacts produced while preparing pruning runs.

Two operational facts follow. The file paths surfaced in viz.py and registries.py assume this layout, so if you relocate the data you have to update registries.py. And the clear_cache.py utility exists because the caching is worth clearing: it deletes locally cached copies of blobstore files, both the Streamlit and viz caches and the tiktoken cache, and you run it when you need to re-fetch fresh artefacts.

That last point is the practical workflow. Because the dashboard caches aggressively to stay responsive, an artefact that has been regenerated upstream will not appear until you clear the cache. The second utility, per_token_viz_demo.py, holds minimal examples for token-level visualisations.

## Model names, sweep ids and what the pruning actually did

The model list is a catalogue of the experiments rather than a version history, and the names encode the experiment.

csp_yolo1 is the model used in the single_double_quote qualitative results, a 118M total parameter model. The README flags it as somewhat older and trained with methods not exactly the same as in the paper, in particular a method for training with multiple L0 values at the same time, so it is a special case rather than the reference. csp_yolo2 is the model used in the bracket_counting and set_or_string_fixedvarname qualitative results, at 475M total parameters.

The csp_sweep1_* models produced the figure 3 results, and the name encodes the model size as an expansion factor relative to an arbitrary baseline size, the weight L0, and the activation sparsity level. There are also csp_bridge1 and csp_bridge2, described as bridge models, with csp_bridge1 being the one used to obtain the results in the paper. Finally dense1_1x, dense1_2x and dense1_4x are dense models trained on the same dataset at one, two and four times the width, which is the control group for the sparse models.

The sweep ids record the pruning configuration, and this is where the honest documentation is. prune_v2 is 256 CARBS iterations at batch size 16, described as a very old, unpublished pruning algorithm, targeting a fixed k rather than a fixed target loss. prune_v3 is 256 iterations at batch size 64 over 32 epochs, an old unpublished algorithm targeting a fixed target loss. prune_v4 is 768 iterations at batch size 64 over 48 epochs, the published algorithm. prune_v5_logitscaling is 256 iterations at batch size 32 over 32 epochs, the published algorithm with logit scaling.

So half the available sweep ids are historical configurations that were not the paper's method, and anyone comparing results across sweeps needs to know which of the four they are looking at. One other gap is worth naming. The README says all models used to obtain the paper's results are released and points to registries.py for the list, then states that exact training hyperparameters can be found in a link labelled todo. That placeholder is still in the README, so the hyperparameters are not actually published there.

## What this release is not, and where the limits are

Three limits deserve stating plainly, because each of them is a place where a reasonable expectation is wrong.

First, the GPT implementation is not a general-purpose model. It is described as lightweight and exists so that these particular checkpoints can be run and probed without a larger framework between you and the activations. If you want to train a transformer, fine, but this is not the code for that. The test fixture with a single layer and a vocab of 16 makes the intent obvious.

Second, the training side is not here. The pruning sweep ids reference CARBS iterations and epoch counts, and the model names reference L0 and activation sparsity levels, so the pipeline that produced this data involved training with sparsity, pruning, and mask extraction. The repository gives you the results and the tooling to inspect them, not the tooling to reproduce them end to end. The hyperparameter link being marked as a todo is the visible symptom of that.

Third, the size of the artefacts. Models at 118M and 475M parameters, plus per-sweep visualisation payloads with activations and samples, means the data set is large and it lives on a public blob store. The caching in the app exists for a reason, and a network-restricted environment will find the app slow or unusable until the files are already cached locally.

There is also a licence wrinkle to check before you build on it. The repository is published under Apache-2.0, which is permissive and lets you use the code in a commercial product with attribution. The pyproject.toml, however, lists the classifier as License :: OSI Approved :: MIT License. Those two statements disagree, and which one governs depends on the LICENSE file rather than on the packaging metadata. That is a small thing to resolve now rather than a small thing to resolve when you ship.

## Conclusion

Adopt circuit_sparsity if you want to reproduce or extend the sparse circuit analysis in Gao et al. 2025, and you are working in Python with the released checkpoints. Do not adopt it as a general GPT implementation, since the model code is described as lightweight and is only meant to load these particular checkpoints, and do not expect the training hyperparameters, since the README marks their location as a todo. Verify first that your Python is at least 3.10, that the blobstore assets are reachable from your network, and that the model directory you point load_model at contains the beeg_config.json and final_model.pt pair the loader expects.

## FAQ

### What is the openai/circuit_sparsity repository?

It is the open-source release accompanying Gao et al. 2025 on understanding neural networks through sparse circuits. It provides code for running inference plus a Streamlit dashboard for interacting with the task-specific circuits found by pruning.

### How do I install and launch the circuit_sparsity visualizer?

Run pip install -e . from the project root, then start the app with streamlit run circuit_sparsity/viz.py. You need Python 3.10 or newer, and the app loads data from the openaipublic webpage and caches it locally.

### How do I get the activations out of a model in circuit_sparsity?

Use the hook_recorder context manager from circuit_sparsity.inference.hook_utils around a forward pass. The result is a dict keyed by names such as 0.attn.act_in, where the leading number is the layer index.

### Where are the circuit_sparsity models and visualisation data stored?

Under the public blobstore at openaipublic.blob.core.windows.net/circuit-sparsity, with models/<model_name>/ holding beeg_config.json and final_model.pt, and viz/<experiment>/<model_name>/<task_name>/<sweep>/<k>/ holding viz_data.pkl. The paths in viz.py and registries.py assume this layout.

### Which pruning sweeps in circuit_sparsity reproduce the published results?

The README lists prune_v4 as the published algorithm, with 768 CARBS iterations, batch size 64 and 48 epochs, and prune_v5_logitscaling as the published algorithm with logit scaling. The prune_v2 and prune_v3 ids are marked as old and unpublished algorithms.

## Sources

- [Issues](https://github.com/openai/circuit_sparsity/issues)
- [License: Apache-2.0](https://github.com/openai/circuit_sparsity/blob/main/LICENSE)
- [openai/circuit_sparsity on GitHub](https://github.com/openai/circuit_sparsity)
- [Project website](https://openai.com/index/understanding-neural-networks-through-sparse-circuits/)
- [README](https://github.com/openai/circuit_sparsity/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/openai-circuit-sparsity