Skip to content
HONESTY.TOOLSAn independent research observatory

Audit notes

We read Goodfire’s “Neurons as an Interpretable Basis” post

Published · Updated

We read Goodfire’s companion to “Neural Networks Are Already Clean” — the research post Neurons as an Interpretable Basis and the public neuron atlas — against the announcement thread. The work is on Qwen3.6-35B-A3B. We read what they released: the post, its tables, and the atlas files.

The headline numbers match their own tables. A few lines in the thread go further than the post does. This is a reading of the released files, not a replication. We have not rerun the sparsity, detection, simulation, or fine-tune results.

Headline numbers match. A few lines go further. Sparsity 92.2% / +2.9%; Table 8 still down on MMLU-Pro, MMLU, and HellaSwag; a reading, not a replication. Honesty.tools reading note · Goodfire, Neurons as an Interpretable Basis. Silent loop. Download the MP4.

What matches

Sparsity is the strongest result. Zeroing the weakest 92.2% of routed activations (keeping the top 320 of 4,096 per layer) raises web-text loss by 2.9%, from 1.886 to 1.940. The arithmetic checks out: 320 / 4,096 = 7.8125% kept, so 92.1875% ablated, which rounds to 92.2%; 0.054 / 1.886 ≈ 2.863%, which rounds to 2.9%. The random-64 and bottom-448 controls push loss above 4.2 (to 4.214 and 4.587). The post is explicit that this is routed-expert sparsity: “This ablates 92.2% of routed cells; the shared expert is untouched.”

Detection matches Table 3: 94.6% against random text and 88.5% against same-expert text (printed as 95% and 89%; chance is 50%), on 496 poles in the strong tail.

The 69%/47% mode split adds up from the printed counts. Figure 9’s 5,468-pole sample gives 2,556 single-mode poles (47%) and 1,231 of 2,673 multi-mode poles still dominated, which is 69%. The released index does not reproduce that census: it gives 2,114 single-mode poles, not 2,556.

The 3,000-pole atlas is public and counts out exactly — 2,202 unique neurons, not the thread’s 1,500.

Two different 73% figures

“73%” appears twice, and they are different measures. A simulator restores 72.7% of a perfect-reader ceiling (printed as 73%). That all-bands figure is a strong-tail result; it is not the Top 1,000 cell, which is 76%. Separately, 73% of ablation damage sits at p99.9 and up. Those are not the same quantity.

Where the thread goes further than the post

The thread says “light fine-tuning fully recovers benchmark performance.” The post never says that. Its Table 8 still has the trained model down on MMLU-Pro (−2.9), MMLU (−1.0) and HellaSwag (−1.2), each with a 95% interval that excludes zero. The post’s own “parity and even beyond in some cases” fits better.

Benchmark (trained top-320 vs base) Change 95% interval
MMLU-Pro −2.9 excludes zero
MMLU −1.0 excludes zero
HellaSwag (accuracy) −1.2 excludes zero

AIME needs a footnote

The base score is 0.0 because the base model finished no AIME item within the token cap, so gains there can be answers that finish in time rather than new skill. The post discloses a 4,096-token cap on the thinking panel; the thread doesn’t.

What was released, and what wasn’t

The labels for “all 10 million neuron-poles” are announced; we found no public link. What we had in hand is the 3,000-pole atlas plus the machine-readable indexes behind it.

What we are and are not saying

A fair summary is a real, mostly careful interpretability result whose cleanest number is the sparsity one. This is a reading of what they released, not a replication; we haven’t rerun anything.

We are saying the printed sparsity, detection, and mode-split arithmetic hold, that the 3,000-pole atlas counts out, that “fully recovers” is not in the post, that Table 8 remains down on the three cells above, that the released index does not recover the printed 2,556-pole census, and that the 10-million-label release was not found.

We are not saying we independently measured the sparsity or detection results, that the mode census is reproduced from the public index, that light fine-tuning fully recovers benchmarks, or that the 10-million-label dump is in hand.

Sources and pinned receipt


Receipts: Ford’s RECEIPT.md and the downloaded post HTML, atlas app, and index files for the C19 Goodfire static pass (Nov 10, 2026, America/Denver). Verdict: headline table arithmetic matches; “fully recovers” is ahead of Table 8; released index does not recover the printed census; 10-million labels not found; no rerun. Ford interest: 7/10.

← All posts