Model or dataset
TransformerLensOrg/TransformerLens avatar
TransformerLensOrg/TransformerLens

TransformerLens: the version field says 0.0.0 and the old API is gone

A library for mechanistic interpretability of GPT-style language models

4,071 stars743 forksPythonMIT

At a glance

What is it?
A mechanistic interpretability library read through its own packaging: a placeholder version in the manifest, a legacy code path still in the tree, an install route for Python versions the manifest forbids, and a numerics default that changed under 4.0.
Who is it for?
TransformerLens fits mechanistic interpretability work on GPT-style models where you need internal activations and the ability to patch them mid forward pass. It does not fit code written against the pre 4.0 API, since HookedTransformer.from_pretrained was removed rather than deprecated, and it does not fit a Python 3.9 environment, since the manifest requires 3.10 while the README still offers a 2.x install line for 3.8 and 3.9.
Can I use it commercially?
Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 2 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • TransformerLensOrg/TransformerLens4,071 ★ · +166
Star history
Repository2026-09-242026-11-15
TransformerLensOrg/TransformerLens3.9k4.1k

166 Stars in the last 52 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

The manifest version is a placeholder and releases come from tags

The project metadata writes version as 0.0.0, which is not a released version of anything, and the release history says otherwise: v4.1.0 on 2026-09-28, v4.0.0 on 2026-09-21, and v3.9.0 on 2026-09-11. The 0.0.0 is a placeholder left in place because the published number comes from the git tag at build time, which is the usual arrangement for a package whose distribution workflow is tag-driven. Reading version out of the metadata tells you nothing, and there is no second literal anywhere in the visible project block to fall back on. The manifest does carry real pins for everything else, and they are strict: torch at 2.6 or newer, transformers at 5.9.0 or newer, beartype at 0.14.1, jaxtyping at 0.3, typeguard in the 4.x band, and huggingface-hub at 0.23.2. numpy is pinned conditionally, at 1.24 or newer for Python 3.10 and 3.11, and 1.26 or newer from 3.12 to just under 3.13, which is the kind of environment marker that saves a numpy ABI failure and also means there is no declared floor for anything above 3.12.

The README offers an install line for Python versions the manifest rejects

The current install is one line:

shell
pip install transformer_lens

Underneath it sits a second instruction with its own heading, for Python 3.8 or 3.9, which pins to the 2.x series:

shell
pip install 'transformer_lens~=2.0'

That older route cannot be reconciled with the manifest, which requires Python 3.10 or later and caps it below 4.0. So the documented escape hatch for old interpreters points at a release line from before the current major version, and installing it means going back to a version of the library whose API the current README no longer describes. That is not a contradiction so much as an unmaintained corner of the page, but it is a real one: a reader on Python 3.9 following the README will get a working install of a two major versions old library and no signal that anything has changed since. The rest of the quickstart, including the TransformerBridge usage, applies only to the current line.

4.0 removed an API rather than deprecating it

The library's central class changed shape between major versions, and the change was not a soft one. The README states that the legacy HookedTransformer.from_pretrained API was removed in TransformerLens 4.0, and points at a dedicated migration guide for moving across. Two things follow. First, there is no deprecation window to lean on: code that calls from_pretrained does not warn, it fails, and there is no version of the new library where it still works. Second, the tree still carries easy_transformer/ at the top level, alongside transformer_lens/, so the older module path remains present in the repository even though the documented entry point has moved. Given that HookedTransformer.from_pretrained is one of the most searched strings associated with this library, the removal is the single most likely thing to break an existing notebook. The migration guide is the thing to read before touching anything else, because the numerics change described below is coupled to the class change.

Activations now match HuggingFace unless you ask for the old numerics

This is the substantive behavioural change in 4.0 and it is easy to miss because nothing raises an error. TransformerBridge is the recommended path and supports the model inventory the README quotes as more than 15,000 models across more than 140 architecture families, with the full list held in a JSON file inside the package at transformer_lens/tools/model_registry/data/supported_models.json. By default the bridge preserves raw HuggingFace weights, which means logits and activations match what HuggingFace itself produces. It explicitly does not match legacy HookedTransformer behaviour, which folds LayerNorm and centres weights by default. Call bridge.enable_compatibility_mode() after booting to get the old numerics back. The practical hazard is silent: an analysis rerun on the new default produces different numbers from the same model with no warning, and any figure in a paper or internal report does not indicate which convention produced it. The compatibility call has to be added deliberately, and it sits on the booted bridge rather than in configuration.

Gated models need a token, and the example file explains how to supply one

Model access is a separate concern from the library itself. The quickstart says gated models, listing Llama, Mistral and Gemma among them, require HF_TOKEN in the environment, and points at an environment variables page for the full set. The repository ships a template file that says more than the README does. It is an example environment file intended to be copied to a gitignored .env and sourced with the shell rather than read by the library at import time, which is a detail worth knowing since it means the library never loads that file on its own. The token itself is a HuggingFace access token created in the account settings. Two optional switches are documented in the same file and both are off by default. One retries HuggingFace Hub 429 responses, and the comment notes it only matters for ad hoc scripts such as the docs build and the demo notebooks, because the pytest suite already enables retries unconditionally through a fixture and CI sets the variable. The other allows the MPS device on macOS, off by default to avoid divergence from CUDA and CPU results.

The dependency list explains itself in comments

Several dependency lines carry a reason next to them, which is unusual and useful. accelerate is required for a function called align_module_device, with the parenthetical naming disk offload as the reason. jaxtyping carries a version floor and the reason for it: the 0.2.x import hook uses ast.Str, which was removed in Python 3.14, and it emits a deprecation warning for every instrumented module until that is addressed, so 0.3 or newer is required to avoid both. typeguard is capped below 5. The extras are narrower. evals pulls lm-eval and pins chardet below 6, and the comment explains that chardet 6 or newer triggers a compatibility check warning in requests, so the pin is a workaround to be removed once requests raises its own cap. inspect pulls inspect_ai, lit pulls lit-nlp, and further groups exist including a vLLM driver with its own validated version band. The pattern is consistent: where a version is constrained, the constraint is a workaround for something outside this project.

The evidence base is a citation list rather than a benchmark

The README argues for the library's usefulness through the work done with it, and it is worth reading what kind of work that is. The research gallery includes progress measures for grokking, sparse probing for finding neurons in a haystack, automated circuit discovery, a linear emergent world representation in Othello GPT, a circuit for Python docstrings in a four layer attention only transformer, a toy model of universality, neuron representation quantification in large language models, and the tuned lens for eliciting latent predictions. Alongside those sit user contributed examples, including an induction heads phase change replication and a set of decision transformer interpretability scripts that use the library for attribution and ablations. None of these is a performance comparison of the library against another, and no installation numbers, supported model counts other than the 15,000 and 140 figures, or timing claims appear. The framing is that the field is young, the entry bar is low, and there are open problems available to work on.

Editorial conclusion

TransformerLens fits mechanistic interpretability work on GPT-style models where you need internal activations and the ability to patch them mid forward pass. It does not fit code written against the pre 4.0 API, since HookedTransformer.from_pretrained was removed rather than deprecated, and it does not fit a Python 3.9 environment, since the manifest requires 3.10 while the README still offers a 2.x install line for 3.8 and 3.9. Before migrating, decide whether you want raw HuggingFace numerics or the folded, centred ones that compatibility mode restores, since the default changed and results are not comparable across that line without saying which was used.

Frequently asked questions

What does TransformerLens do?

It loads GPT-style language models and exposes their internal activations, letting you cache any internal activation and add functions that edit, remove or replace those activations while the model runs. The library is described as an implementation of transformers tailored for mechanistic interpretability, which is reverse engineering the algorithms a trained model learned from its weights.

What is TransformerLens and how do I install it?

It is a PyTorch library for mechanistic interpretability of GPT-2 style models, licensed MIT. The install is pip install transformer_lens, and the manifest requires Python 3.10 or later with torch at 2.6 or newer and transformers at 5.9.0 or newer. The package name on PyPI is transformer_lens.

Why does HookedTransformer.from_pretrained no longer work?

That API was removed in TransformerLens 4.0 rather than deprecated, so existing code calling it fails instead of warning. TransformerBridge is the recommended replacement, with a migration guide published for moving across the major version.

Do TransformerLens activations match HuggingFace?

By default, yes. TransformerBridge preserves raw HuggingFace weights, so logits and activations match HF rather than legacy HookedTransformer, which folds LayerNorm and centres weights by default. Calling bridge.enable_compatibility_mode() after booting restores the HookedTransformer equivalent numerics.

How do I load a gated model such as Llama with TransformerLens?

Gated models including Llama, Mistral, and Gemma require HF_TOKEN in the environment. The repository ships an example environment file to copy into a gitignored .env and source from the shell, and it also documents optional switches for retrying Hub 429 responses and for allowing the MPS device on macOS, both off by default.

Official sources

  1. License: MIT
  2. Project website
  3. README
  4. Releases
  5. TransformerLensOrg/TransformerLens on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/transformerlensorg-transformerlens.svg)](https://hysenlabs.com/projects/transformerlensorg-transformerlens)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/transformerlensorg-transformerlens.svg)](https://hysenlabs.com/projects/transformerlensorg-transformerlens)

Was this useful?