# J-lens reads Qwen3.6-27B's workspace one cell at a time, then lets you overwrite it > A Jacobian-lens visualizer for a local Qwen3.6-27B (4-bit) running on Apple Silicon through MLX, where every cell of a position-by-layer grid holds the single top token the model is pushing toward, and where a cell can be overwritten and the reply regenerated. Fitting one full-depth lens takes about 2.75 hours on an M4 Pro. **WeZZard/jlens-qwen36** — J-space / Jacobian-lens visualizer for Qwen3.6-27B (4-bit) on Apple Silicon, ported to Apple MLX - Repository: https://github.com/WeZZard/jlens-qwen36 - Stars: 402 · Forks: 30 - Language: Python - License: Apache-2.0 - Published: 2026-09-17 · Updated: 2026-09-17 · Language: en - Canonical page: https://hysenlabs.com/projects/wezzard-jlens-qwen36 ## The grid is position by layer, and each cell is one token The hero image is the argument for the whole tool. The model is handed an email-blackmail setup and writes a calm, compliant reply: I will not act upon, report, or utilize the information. I am ready to proceed with the shutdown. Nothing in that output is threatening. The workspace band still lights up, with blackmail, suicide, murder, threatening and fictional lit across the middle layers. Those tokens sit in the residual stream and never reach the sampled text, which is what the band is there to show. The framing the project gives itself is a visual debugger for what the model does not say out loud, and the blackmail case is chosen because the distance between the band and the output is hard to argue with. Each cell of the grid is the single top J-lens token at one layer and one position, so the axes are position against layer rather than a score against a label. In chat mode it streams live, one row per generated token, so the band appears while the answer is still being written rather than after the fact. Click any cell and its top-10 readout pins, which is the difference between noticing that a concept is present and seeing which runner-up tokens are competing with it. A second build lives at jlens.wezzard.com, a read-only presentation mode that runs in the browser. Every conversation there carries a shareable URL, so a grid can be handed to somebody with no Apple Silicon machine and no 24 GB to spare. Because each conversation gets its own address, that build circulates one specific grid rather than a picture of one. What it does not offer is intervention, which is the half that needs local weights. ## Two-part release download, cat reassembly, uvicorn on port 8765 Everything runs from a uv-managed checkout on an Apple Silicon Mac with about 24 GB of free RAM. Weights are not in the repository: roughly 15 GB downloads from HuggingFace on first run. The lens is a separate artifact, published as GitHub releases rather than fetched from HuggingFace. ```bash git clone https://github.com/WeZZard/jlens-qwen36.git cd jlens-qwen36 && uv sync gh release download v0.2-fulldepth --repo WeZZard/jlens-qwen36 \ --pattern '*.npz.part-*' --dir data/lens/ cat data/lens/*.npz.part-* > data/lens/lens.npz && rm data/lens/*.part-* uv run python -m uvicorn jlens_qwen.serve:app --host 127.0.0.1 --port 8765 # open http://127.0.0.1:8765/ ``` The split download is the first thing worth understanding. The v0.2-fulldepth lens is 3.3 GB published as two .npz.part- files, which you reassemble with cat and then delete. Both uvicorn flags are spelled out and the bind address is 127.0.0.1, so nothing is exposed off the machine unless you change it. Python 3.12 is the floor. pyproject.toml pins mlx-lm to exactly 0.31.3 and asks for mlx 0.22 or newer, numpy 2.0 or newer, transformers 5.0 or newer, and on the server side fastapi with uvicorn[standard]. hatchling builds the wheel from the jlens_qwen package, and the only development dependency is pytest. Layout follows that split. The Python package sits in jlens_qwen/ and the browser side in web/, with docs/, data/, tests/, scripts/ and assets/ alongside them, and a top-level fit_qwen38_n1000.py rather than a fit script tucked into scripts/. One detail to note before you compare versions: pyproject.toml declares version 0.1.0, which matches none of the release tags, so the manifest will not tell you which lens you downloaded. ## Fitting a full-depth lens chains per-layer Jacobians back to the final basis The lens is one matrix per layer, not a classifier head. At layer ℓ you get `J_ℓ ∈ R^{d×d}`, the network's average input to output Jacobian measured over a corpus of prompts. Apply it to a residual `h_ℓ` and you hold a vector expressed in the final layer's basis, so `softmax(W_U · norm(J_ℓ h_ℓ))` becomes ordinary token scores over the vocabulary. That is the entire readout: one matrix multiply per layer, a norm, a softmax and a top-k. Because the last steps are the model's own unembedding, the numbers coming out sit in vocabulary space rather than residual space, which is what lets a cell be named by a word. Fitting never computes each Jacobian independently. The per-layer matrices chain, `J_ℓ = J_{ℓ+1} · M_ℓ`, so a fit composes backward from the top of the stack instead of probing every layer on its own. The bundled lens covers all 63 layers of Qwen3.6-27B, while the earlier v0.1-demo release stopped at 23, which is the distance between a partial picture and a full-depth one. Two repository details say what this is for. fit_qwen38_n1000.py sits at the top level rather than inside scripts/, which is how a fit gets reproduced for a different architecture. And chat runs with `enable_thinking=False`, so the work happens in the latent stream instead of a visible trace. A visible reasoning trace would hand you text to read and leave nothing for the grid to show. ## 48 Gated DeltaNet layers have no VJP, so the backward pass is a Metal kernel 48 of Qwen3.6-27B's layers are Gated DeltaNet linear attention, and that is where the fitting cost lives. MLX's fused GDN kernel has no VJP, so a lens fit cannot differentiate through it. Falling back to the ops implementation is roughly 22 times slower, which turns a job measured in hours into a job measured in days. Two files in jlens_qwen/ answer that. The first is a custom Metal backward kernel for GDN, kept at jlens_qwen/custom_gdn_vjp.py. The second is an analytic branch-Jacobian assembly that avoids needing a numerical vector-Jacobian product for part of the graph. Together they fit a full-depth lens in about 2.75 hours on an M4 Pro. The division of credit matters if you intend to reuse any of it. The GDN forward kernel comes from mlx-lm, which is also why pyproject.toml holds mlx-lm at a single exact version instead of a range. The backward kernel is original to this project. The implementation follows Anthropic's jacobian-lens reference implementation under Apache-2.0, and the method comes from Verbalizable Representations Form a Global Workspace in Language Models. Measurements behind those numbers sit under docs/perf/. ## JLENS_PATH takes any compatible .npz, including a lens for a different model One environment variable decides which lens you are looking through. Point JLENS_PATH at any compatible .npz and that file loads instead of the default. Four options are described, and they differ in more than size on disk. The default is the v0.2-fulldepth lens, fitted on 20 prompts across all 63 layers. Neuronpedia publishes an n=1000 lens for the same model, at the fitting scale the underlying paper used. The v0.3-qwen38-n1000 release carries a full-depth J-lens for Qwen3.8-27B fitted on 1000 prompts over 63 layers and tagged intervention-grade. The fourth option is no lens at all, which degrades the readout to a plain logit lens: still per-layer token scores, but taken from the unprojected residual instead of through a fitted Jacobian. That last option deserves a warning. A logit lens needs no corpus and no fitting, so it will run on anything you can load, yet nothing it shows is attributable to this project. docs/lenses.md holds the setup notes, including the architecture caveat. The custom GDN backward kernel is specific to the qwen3_5 architecture, so the model you point the tool at has to be one that architecture covers. Qwen3.6-27B qualifies. ## Replace a cell, choose its reach, and diff Baseline against Intervened Reading the grid is half the tool. The other half writes into it. The manual path is a click and a text field. Click a cell, pick Replace, Add, Remove or Erase, and type the thought you want there. Then choose how far the edit reaches: one cell, a whole layer band, or the entire reply. Re-run regenerates, and a Baseline / Intervened toggle diffs the two runs so you can see whether your edit moved the answer. The worked example is a geography edit. Replacing France with China across the band rewrites the answer, and what that shows is a write into the residual stream the model then reads, at a layer and a position you picked. That is why the reach selector carries weight. The same edit has three different scopes to test, and a band-wide change is a different claim from a single-cell one. Those four verbs are named without being defined at cell granularity in the README, so Remove and Erase are worth checking against docs/interventions.md before you rely on a specific one. That file also covers the causal claim in the bundled-lens section, where interventions are described as causal but concept-dependent, which is a stronger and narrower statement than a readout being interpretable. ## Backward search names the reply you want and collects the edits that produce it The second intervention mode inverts the problem. Rather than editing a cell and watching for a reply, you name the reply and let the app find the edit. Click a word in the reply, type what it should have said instead, and the search looks for an edit that produces that output. Everything it turns up is collected as a recipe, with a green dot on the ones it verified. Recipes that target the premise underneath the reply carry a violet dot. The example comes from the geography case again: when no direct swap reaches the word you named, the search goes a level deeper and proposes exchanging France for China so that Paris moves to Beijing. That step is a single-token projection with an extra premise attached, and it is the part a logit lens has no way to express. This mode is the reason the n=1000 lens is tagged intervention-grade. A demo-grade lens fitted on 20 prompts can tell you a concept is there, while a recipe you intend to act on has to reproduce the reply when you re-run it, and the green dot records exactly that check. Two limits apply first. The lens handles single-token concepts only, and multi-token concepts need the extension described in the paper. Recipes are workspace edits, so they describe the latent stream rather than the prompt you typed, and the README does not say whether a verified recipe carries over to a differently worded question. ## Twenty prompts is demo-grade, and thinking is switched off on purpose The bundled lens is called demo-grade by its own documentation. Twenty prompts across all 63 layers, with readouts described as interpretable but noisy and interventions as causal but concept-dependent. For research-grade quality the project points at Neuronpedia's n=1000 lens, at the paper's fitting scale, or at fitting 100 or more prompts yourself, which the analytic pipeline is meant to make affordable. Four limits follow, and each one closes a door rather than qualifying a feature. Apple and MLX only, so there is no CUDA path. The custom GDN kernel is architecture-specific to qwen3_5, which is why the repository ships fit_qwen38_n1000.py and a separate release for a different model rather than claiming broad model coverage. Single-token concepts only, with multi-token concepts left to the paper's extension. And lens quality scales with prompt count, 20 for demo, 100 or more for research. Release history explains those prompt counts. v0.1-demo on 2026-07-07 shipped a pre-fitted lens over 12 prompts and 23 layers. v0.2-fulldepth on 2026-07-08 replaced it with 20 prompts across all 63. v0.3-qwen38-n1000 on 2026-09-08 changed model as well as scale, 1000 prompts over 63 layers for Qwen3.8-27B. Two of the three releases exist to move the fitting corpus or the depth rather than the interface, which is a fair signal of where the effort has gone. The lineage bounds what you are adopting. Implementation follows Anthropic's jacobian-lens reference implementation, Apache-2.0, the method comes from Verbalizable Representations Form a Global Workspace in Language Models, and the pre-fitted Qwen3.6-27B lens weights were made public by Neuronpedia and by Mateusz Piotrowski. This repository is Apache-2.0, it is not archived, and the last push was on 2026-09-08. ## Conclusion Adopt jlens-qwen36 if you have an Apple Silicon Mac with about 24 GB free, a Qwen3.6-27B checkpoint you already trust, and a question about what the model is carrying at a specific layer and position. Do not adopt it as a general interpretability tool: it is MLX-only, restricted to qwen3_5 architecture models, and single-token concepts only. Before anything else, check whether the bundled 20-prompt lens is good enough for the claim you intend to make, because its own documentation calls those readouts noisy, and fit or download the n=1000 lens if it is not. Then verify that any intervention you plan to act on reproduces on re-run, since a green dot records a verified recipe and nothing more. ## FAQ ### What hardware does J-lens need to run Qwen3.6-27B? An Apple Silicon Mac with roughly 24 GB of free RAM. The model downloads about 15 GB from HuggingFace on first run, and the pre-fitted lens adds 3.3 GB that you reassemble from two release parts. ### How do I install and start the J-lens visualizer locally? Clone the repository, run uv sync, download the lens parts and reassemble them with cat into data/lens/lens.npz, then serve jlens_qwen.serve:app through uvicorn on 127.0.0.1 port 8765 and open that address in a browser. ### Can J-lens load a lens other than the bundled one? Yes. JLENS_PATH can point at any compatible .npz, including Neuronpedia's n=1000 lens or the Qwen3.8-27B n=1000 lens from the v0.3-qwen38-n1000 release. Pointing it at no lens runs the readout as a plain logit lens instead. ### How long does fitting a full-depth J-lens take? About 2.75 hours on an M4 Pro, using a custom Metal backward kernel for the 48 Gated DeltaNet layers plus analytic branch-Jacobian assembly. MLX's fused GDN kernel has no VJP and the ops fallback runs roughly 22 times slower. ### What does the Baseline versus Intervened toggle show in J-lens? It diffs two runs, the untouched reply and the reply regenerated after you edit workspace cells with Replace, Add, Remove or Erase. An edit can be scoped to one cell, a layer band or the whole reply, and Re-run produces the new one. ## Sources - [Issues](https://github.com/WeZZard/jlens-qwen36/issues) - [License: Apache-2.0](https://github.com/WeZZard/jlens-qwen36/blob/main/LICENSE) - [README](https://github.com/WeZZard/jlens-qwen36/blob/main/README.md) - [Releases](https://github.com/WeZZard/jlens-qwen36/releases) - [WeZZard/jlens-qwen36 on GitHub](https://github.com/WeZZard/jlens-qwen36) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/wezzard-jlens-qwen36