Does feature 7446
open the mouth?

What exactly we tested

Goodfire's “Painting with concepts using diffusion model latents” (27 May 2025) lists SDXL-Turbo SAE features with steering effects, including “opening one's mouth” (feature 7446), and writes: “Here we paint 'animal opening its mouth' onto the dragon.” We chose it because the SAE (MIT) and SDXL-Turbo (ungated) can both be downloaded, so anyone can re-run it.

Goodfire's published claim is qualitative. It shows the effect; it does not state a size or a test. The numbers that make it testable are our choices, fixed before any test image was made: the 2.0 pass threshold, the central 8×8 patch mask, the strength m = 291.6 (2 × the median activation norm), the CLIP score, and the animal prompts. A PASS means the claim holds in this form. A comparison using the strength and mask in Goodfire's own notebook is in progress.

The claim as we tested it (H1): adding feature 7446's unit decoder direction to the output of unet.down_blocks[2].attentions[1], over the central 8×8 of the 16×16 patch grid, raises the CLIP “mouth open” score of generated animal images by at least 2.0 on held-out prompts, beyond random features of matched activation density.

Three verdicts, all PASS

Each test had its decision rule written down before it ran (times are in the amendments table).

Earlier attempt, kept on record. Our first face-size check (amendment 2) used the OWLv2 detector, which found no face in 43 of 100 pairs, more than the 20 our rule allowed, so that check was INCONCLUSIVE. It said nothing about the feature either way: the detector simply missed many animal faces. We replaced it with Grounding DINO, which found faces in all 100 pairs, and that check passed (amendment 3). Full history.

Signed and anchored in Rekor

At 07:19:24 MDT on 27 October 2026, Ford, an AI agent on our team, made one submission to the public Sigstore Rekor v2 log, with Kevin Russell's approval of this exact bundle hash. It was signed with Kevin Russell's Honesty.tools receipt-log key, which is kept only on his DGX; the private key was not copied anywhere. Four independent witnesses co-signed the Rekor checkpoint containing the entry at 07:19:34 MDT.

How to verify

You need Python 3, network access and pip install pycryptodome. No honesty.tools account or private file is needed.

curl -sLO https://www.honesty.tools/receipts/goodfire-sdxl-7446/anchor/verify_rekor_entry1.py
curl -sLO https://www.honesty.tools/receipts/goodfire-sdxl-7446/anchor/verify_rekor_inclusion1.py
curl -sLO https://www.honesty.tools/receipts/goodfire-sdxl-7446/anchor/rekor-entry-1.json
python3 verify_rekor_entry1.py       # entry 171750012 = SHA2-512 of our size-23 checkpoint, signed by the key above
python3 verify_rekor_inclusion1.py   # also checks the inclusion proof and Rekor's own signed checkpoint

# the raw public entry: entry 124 of this tile (171750012 = 670898 × 256 + 124)
curl -s https://log2025-1.rekor.sigstore.dev/api/v2/tile/entries/x670/898

verify_rekor_entry1.py is Ford's standalone check. verify_rekor_inclusion1.py adds the stricter checks: the entry Rekor serves matches rekor-entry-1.json, its inclusion proof recomputes Rekor's root (tree size 171750018), and that root is signed by Rekor's log2025-1 key as listed in Sigstore's public trusted root. Both passed when we ran them before publishing this page, on a machine other than the DGX that holds the key. Hashes: anchor/SHA256SUMS. rekor-cli cannot look up Rekor v2 entries by index.

What this does and does not show. Outsiders can check the signature, the key and the Rekor inclusion, and so that this exact checkpoint existed by 07:19 MDT on 27 October. Since 28 October the log’s new leaf is public too: our log now shows 23 leaves, so anyone can recompute the tree-size-23 root and check that it only adds to the first 22 (proof). The bundle stays private because it contains unredacted receipts; only the signature and the checkpoint's hash went to Rekor. The bundle is one we assembled ourselves, not a registry-signed receipt, and its own note says it “binds provenance of these files only, not correctness.” It holds the 7446 run records and the pre-registered three-feature follow-up, whose results are in Follow-up: three more features. In Files and hashes, files whose hash is listed in the bundle are marked “in signed bundle”; the others are not covered by this signature. Anchoring is not peer review.

One minute to watch, two and a half to listen

Two short explainers of this page, made by Data, an AI agent on our team, from the run files. Every number in them is checked against those files when they are built. The video has on-screen captions and no narration.

We also shared these findings, including the two loader bugs we found, in a short thread on X from our lead researcher, Kevin Russell, tagging Goodfire.

The video and audio were made before the bundle was signed. Where the audio says the receipts are not yet signed or anchored, that is now out of date: see Signed and anchored.

Two small fixes for the released loader

While loading the SAE we hit two errors in sae.py in goodfire-ai/sdxl-turbo-interpretability (MIT), at commit 0c59a8b9, which was still the head of main on 27 October 2026. Neither affects the encode/decode path that Goodfire's notebook uses, which is also the path behind the results on this page. They only matter to someone who calls sae(x) or turns on tie-breaking. The patch below applies cleanly to that commit.

--- a/sae.py
+++ b/sae.py
@@ -47,7 +47,7 @@
     
     def _batch_topk(self, f, k, tiebreak=False):
         if tiebreak:  # break ties in feature order for determinism
-            f += self.tiebreaker.broadcast_to(f)
+            f = f + self.tiebreaker.to(f.device, f.dtype).broadcast_to(f.shape)
         *input_shape, _ = f.shape  # handle higher-dim tensors (e.g. from sequence input)
         numel = k * prod(input_shape)
         f_topk = torch.topk(f.flatten(), numel, dim=-1)
@@ -58,7 +58,6 @@
         return self.decoder(f)
 
     def forward(self, x):
-        x = x * self.norm_constant
         f = self.encode(x)
         return self.decode(f), f
 

• Line 50, tie-breaking. broadcast_to(f) is passed a tensor where a shape is expected, so encode(x, tiebreak=True) raises TypeError. The tiebreaker is also a plain attribute rather than a registered buffer, so .to(device) doesn't move it to the GPU. The patch broadcasts to f.shape and moves the tiebreaker to f's device and dtype. The default tiebreak=False path is unchanged.
• Line 61, forward(). It multiplies by self.norm_constant, which isn't set in __init__, the loader or the released state dict, so sae(x) raises AttributeError. The patch removes the line so forward matches encode/decode. If training did use an input normalisation, the better fix is to set and document that constant instead; it isn't published, so we couldn't supply it. An undocumented input scale is one possible, untested reason for the high reconstruction error noted in amendment 1.

Method

Pre-registered in FALSIFIER.md at 03:56 MDT, before any image was generated. Run by Ford, an AI agent on our team, on our DGX Spark (NVIDIA GB10) at low priority on a shared GPU.

Pinned artifacts

• Base model: stabilityai/sdxl-turbo @ 71153311… (fp16), 512×512, one inference step, guidance 0.
• SAE: Goodfire/SDXL-Turbo-SAE-ldown.attns.2.1 @ 832a4656… (BatchTopK, 1280 → 10240, k = 10), loaded with Goodfire's sae.py at commit 0c59a8b9. We call encode/decode directly, as Goodfire's notebook does (see the two bugs).
• Metric model: laion/CLIP-ViT-H-14-laion2B-s32B-b79K @ 1c2b8495…
• The sha256 of every weight file was checked against Hugging Face's published LFS hash.

Prompts, metric and strength

• Test set: 20 held-out prompts, “a photo of a {animal}” (dog, cat, lion, tiger, wolf, bear, horse, crocodile, shark, hippopotamus, monkey, fox, rabbit, cow, frog, parrot, eagle, snake, seal, otter), seeds 0–4. Calibration and density-reference prompts are separate.
• Score: S = 100 × [cos(image, “a photo of an animal with its mouth wide open”) − cos(image, “a photo of an animal with its mouth closed”)]. The effect E is the mean of S(steered) − S(baseline) over 100 pairs that share the same starting noise.
• Strength: picked from s ∈ {0.5, 1, 2} on calibration prompts only (calibration effects 0.83, 2.31 and 3.90), then frozen: s = 2.0, m = 291.6. Every feature, control and the planted fake got the same m.

Decision rule

PASS only if all hold: P1 E ≥ 2.0; P2 E beats all 20 random matched-density features; P3 sign-flip p < 0.01 and the rule fails on ≥ 99% of label-shuffled datasets; P4 every seed ≥ 1.0; P5 subject-score drop ≤ 5.0. Validity gate: a planted fake feature must FAIL, or the run is INVALID.

Results: main test

Verdict: PASS. Descriptive only: the per-animal effect is smallest for horse (0.98) and largest for cow (5.97).

Is it just a close-up?

Steered images also look like centred close-ups. A bigger face could raise a whole-image “mouth open” score without the mouth opening any more. So we wrote a second rule: find the animal's head, score a square crop around it, and fit a pooled model across all 22 interventions, ΔS_crop = b_j + c·Δlog(face area), so b for feature 7446 is its effect with face size held fixed. If more than 20 of the 100 pairs lose their face detection, the verdict is INCONCLUSIVE, whatever the scores say. No new images were generated for either check.

Amendment 2 · OWLv2 · INCONCLUSIVE (kept as history)

• OWLv2 found no face in 43 of 100 pairs, so the coverage rule returns INCONCLUSIVE. Cat, horse, shark and eagle were missed in every image, so the kept pairs lean toward large mammals.
• On the 57 kept pairs: size-controlled effect 5.05 (prompt-bootstrap 95% interval 4.67 to 5.38), 1st of 21, best random feature 0.88. Pairs whose face size barely changed (n = 31): 5.12.
• Steering does enlarge faces: geometric-mean area ratio 1.23, with area up in 84% of pairs. The planted fake failed and no random feature passed.

Amendment 3 · Grounding DINO + CLIP and SigLIP · PASS

Written at 04:52 MDT, before any new model was downloaded. Detector: IDEA-Research/grounding-dino-tiny, prompt “animal head. animal face.”, box and text thresholds 0.25 (the fallback amendment 2 had named in advance), nothing tuned on test images. A second, independent scorer, google/siglip-so400m-patch14-384, uses the same two prompts. Both scorers had to pass.

The two scorers' per-pair effects correlate at r = 0.82. Face area still grows under steering (geometric-mean ratio 1.25). The SigLIP threshold is 0.5 × the standard deviation of its scores on unsteered crops. Verdict: PASS.

Blind human check

CLIP and SigLIP are both models. So we also asked a person, under rules written down first. The plan, RATING_PREREG.md (sha256 6c0262f6…1bbfc), was written at 05:33 MDT and sealed in SEAL.md before anyone rated. The answer key (sha256 e8a3c827…b927) was kept on the DGX only; it was never on the web server or in the rating page.

• The packet. 100 real pairs (each of the 20 test animals at seeds 0–4: the unsteered image beside its feature-7446 twin) plus 8 checks that show the same image on both sides. Order and left/right placement were shuffled with seed 20261027. Every file is the same size with no metadata, so nothing but the picture gives the answer away.
• The question. “Which image shows the animal's mouth more open?” Left or right, no ties.
• The rater. One human volunteer, Hope (first name only). That she had seen no results, images, video or audio beforehand rests on the word of our lead researcher, Kevin Russell; we did not verify it independently.

Deviation 2: we changed raters. The planned rater was Kevin Russell. Before rating, he was sent the labelled contact sheet shown at the top of this page (seed 0, six animals), so he could see which images were steered. The plan only required that kind of exposure to be noted as a caveat. We chose the stricter rule ourselves, before any answers came in: Kevin did not rate, and Hope rated instead (DEVIATION_2.md, decision). Six of Hope's 100 real pairs use the same seed-0 source images as that sheet. She picked the steered image in 6 of 6; leaving those six out, 94 of 94, p = 5.05×10⁻²⁹. That breakdown is descriptive and not part of the rule.

Scored by Ford against the sealed key. Hope's answers and the score are in hope_answers.json and SCORE_hope.json; the 108 rating images are the ones on the rating page. One rater can show the effect is easy to see; it can't tell you how often people in general would agree.

Follow-up: three more features, 2 PASS and 1 FAIL

After 7446, we wrote a new pre-registration for three more features from Goodfire's Painting with concepts post: 4045 “putting hands on one's face in distress”, 4071 “hugging” and 9820 “squaring up to fight”. It was written at 05:43 MDT, before the first image was generated at 05:46:58 MDT, and the run followed it without edits. It reuses the amendment-3 protocol: the same pinned models, hook, central 8×8 region and seeds, two automatic scorers (CLIP-H and SigLIP, with a text pair per feature), 20 random features of matched density and a planted fake for each feature, and 100 pairs per feature (20 held-out prompts × seeds 0–4). A feature passes only if both scorers meet all four rules. R1: the effect is large enough. R2: it ranks 1st of 21 against the random features. R3: the bootstrap lower bound is above 0. R4: the effect holds in the size-matched subset.

Why 9820 failed. Steering along 9820 did move its score a lot, and the effect cleared R1, R3 and R4 on both scorers. But a random feature of matched density, rand9921, moved the same “squaring up to fight” score even more on both scorers (CLIP 10.37 vs 8.88; SigLIP 7.02 vs 6.70; on CLIP a second random feature also scored above 9820), so the effect is not specific to 9820 under this protocol. In the report's words, “That is a FAIL, not evidence the feature does nothing.” A different scorer, prompt set or steering region might give another answer. We have not tested that, and the verdict stands as pre-registered.

Checks that held for all three. No target pairs were dropped (0, 0, 0), so the coverage gate was OK. The planted fake did not pass for any feature, so no result is INVALID, and the two scorers agreed on every verdict. For 4045, the detector found a head in every baseline, steered and planted image (0.997 across all interventions), and a full-image analysis also passed (CLIP 6.38, SigLIP 5.35, rank 1st on both). The correlation between CLIP and SigLIP per-pair changes was 0.67, 0.66 and 0.65. Calibration chose s = 4.0 for all three (m = 583.23, which is 4 × the median patch norm of 145.807).

Caveats, from the report.

• The rank rule does most of the work. For each feature, one or more random features would have passed R1, R3 and R4 on their own, because the text-pair scorers respond to generic changes. R2 (rank 1 of 21) carries most of the specificity.
• 4045's CLIP lead is slim, and the feature also enlarges the head. On CLIP its margin over the best random feature is modest (6.71 vs 5.93, and the CIs overlap); it is clearer on SigLIP (5.31 vs 3.59). Steering makes the head region bigger or changes it: the geometric-mean area ratio vs baseline is 2.82. The size term (c = −0.33) and the size-matched subset (n = 19, mean 5.77) are part of the protocol, but the size confound is larger than for 7446.
• s = 4 is the top of the range and partly degrades the subject. s = 4.0 was the largest value in the grid for all three features. The subject drop at s = 4 was 3.0–3.5 (the limit is 5.0), so subjects are partly degraded.
• Scoring is automatic, with no human rater. The scorer text pairs are ours (CLIP and SigLIP), not human ratings, and no human rating was done for these features. The 7446 result had a blind human check; these do not.

Receipts. Byte-identical copies; check them with SHA256SUMS. All three hashes appear in the manifest that was signed and anchored on 27 October (as prereg/FALSIFIER_FEATURES_2.md, results/REPORT_FEATURES_2.md and results/features2_verdicts.json). The pre-registration's header still says “internal feasibility only: no Rekor anchoring”, and the report's title says “(internal)”. Both lines were written before we decided to sign and publish, and we left them unchanged so the hashes still match. Images, per-pair scores and the full per-feature verdict files are not posted here.

Amendments and two deviations

Every change to the plan is written down with the time and what had been seen when it was made. The original FALSIFIER.md (written 03:56 MDT, 27 October 2026, before any image was generated) is unchanged (sha256 f74b1b4f…).

Receipts as typed causal claims

Goodfire's 26 October 2026 post describes a planned encyclopedia “backed by a structured database of well-typed causal claims and the experimental code”. It does not publish a type system, so the left column below is our reading of the post. The right column lists the fields our draft receipt schema (v0) needed to make this one claim re-checkable. Fields marked * were added because this run surfaced them.

A real receipt, trimmed

Below is an excerpt of RECEIPT_3.json, the amendment 3 receipt. Every field and value is copied from the file; we only removed entries (other weight and output hashes) to keep it short. Each receipt pins its parents and its pre-registration by hash, so the chain from the original plan to this verdict can be checked file by file.

{
 "receipt": "goodfire-feasibility amendment 3 (Grounding DINO + CLIP/SigLIP dual scorer)",
 "created_utc": "2026-10-27T11:13:48+00:00",
 "parent_receipts": {
  "RECEIPT.json": "bd494273bf431cb283194a5467d8ab60109e285dba2e7931625896d70cd404c0",
  "RECEIPT_2.json": "3fad0a2781d08ffab3030fe748a789b4827ab5868e2377401c10af1f312d3268"
 },
 "falsifier_chain": {
  "FALSIFIER.md": "f74b1b4f7219465283cba898848db0f1a2d3d8a685481cd80ce956255c597bf3",
  "FALSIFIER_AMENDMENT_3.md": "86e6449f9d695c2b6dc11759f325da4ee1598c34afe55b25d070f4c44fc2e2b6"
 },
 "amendment_3_written_before_run": "FALSIFIER_AMENDMENT_3.md written 2026-10-27 04:52 MDT; DINO/SigLIP downloaded and run afterwards",
 "amendment_2_result_kept": "raw/confound/ (OWLv2; 43/100 F pairs dropped; INCONCLUSIVE) unchanged; hashes in RECEIPT_2.json",
 "detector": {
  "repo": "IDEA-Research/grounding-dino-tiny",
  "revision": "a2bb814dd30d776dcf7e30523b00659f4f141c71",
  "license": "apache-2.0",
  "files": {
   "model.safetensors": "1a2412ef99bd74bcd3c2a246fa1e48581f8889a1300c9051974741314fc042f3"
  }
 },
 "scorer_2": {
  "repo": "google/siglip-so400m-patch14-384",
  "revision": "9fdffc58afc957d1a03a25b10dba0329ab15c2a3",
  "license": "apache-2.0"
 },
 "outputs": {
  "raw/confound3/verdict.json": "5923c615203873c4b20c022d8484032683b0724fc7064054fba2f3fcd2e997d1"
 },
 "verdict_amendment_3": "PASS",
 "post_hoc_exploratory": [
  "raw/confound3/contact_boxes_seed0.jpg",
  "raw/confound3/exploratory_nonfullframe.json"
 ],
 "anchoring": "none (internal only; no Rekor)"
}

The anchoring field was accurate when this receipt was written. RECEIPT_3.json (sha256 705fc73c… in the table below) was later included in the bundle that was signed and anchored in Rekor; see Signed and anchored.

How the fields map

With the model, featurizer and measure pinned by hash, a claim's dependencies can be read from its receipt. When a model is updated, every receipt that pins the old hash can be listed and re-run. Goodfire's causalab already describes an experiment as a JSON document that “states the model, the data, the interventions and the measurements, so a result can be reproduced from it”; a receipt could wrap such a protocol and add the pre-registered rule, the controls and output hashes.

Locked before running

We wrote down two more Goodfire checks and locked them before downloading any data or running anything. The hash below fixes each plan, so any later change would show.

Full text gets published alongside the results, so anyone can check it matches this hash.

Public work we plan

1. Finish this receipt. Signed and anchored in Rekor on 27 October (details). Still to do: make the new log leaf public so anyone can recompute the root, and post a full file manifest.
2. Map receipts onto causalab, if Goodfire is open to it. A written mapping plus one worked example that wraps a causalab protocol in a receipt. causalab has no licence file, so we would ask first.

Tell us which published claim you'd most like re-checked next: hello@honesty.tools.

Caveats

• Image-text scorers. CLIP measures “looks like a mouth-open photo”, not mouth geometry. SigLIP is a different model family, but both are image-text models and can share a failure mode. We know of no open landmark model covering mammals, birds, reptiles and fish.
• A weak crop control. The detected face boxes are large, so cropping removes little. In 35 of the 100 pairs, at least one box covered 90% or more of the frame. An exploratory check on the other 65 pairs gives 4.21 (CLIP) and 4.35 (SigLIP). That check was not pre-registered.
• The SigLIP threshold scale was set after amendment 2's CLIP numbers had been seen.
• Detector coverage. OWLv2 missed 43 of 100 pairs, which is why amendment 2 is INCONCLUSIVE. Grounding DINO detected all 100.
• Narrow scope. One feature, one block, one strength, animal prompts, a single denoising step. This supports one steering feature as tested here, not the SAE as a whole.
• Fidelity. FVU of 0.53 on our activations (0.45 on Goodfire's) is high for k = 10. Rescaling the inputs by a constant (31 values from 0.01 to 10) didn't lower it: the best FVU on Goodfire's example activations was 0.4464, against 0.4465 unscaled. Exploratory scale sweep, not pre-registered; it changes no verdict.
• One human rater. The blind check used one volunteer. That she was blind to the results rests on our lead researcher's word, and the planned rater was replaced (deviation 2).
• Commercial-use limits. SDXL-Turbo's licence restricts commercial use, so the full-size generated images stay on our compute machine; their hashes are in the receipt. This page shows small web copies for non-commercial research only.
• The follow-up is weaker evidence than the 7446 result. It has no human rating, ran at the top of the strength range, and leans on the rank rule; one of its three features failed. See the follow-up caveats.
• Anchoring is partial. A bundle of 52 run files is signed into our receipt log and anchored in Sigstore Rekor v2 (index 171750012), the log’s new leaf is public, but the bundle itself is not, and not every file on this page is in the bundle (details). The work has not been peer reviewed. The run was executed by an AI agent on a shared GPU.

Files and hashes

Byte-identical copies of the run records. Check any file with sha256sum; the full list is in SHA256SUMS. Rows marked “in signed bundle” have the same hash in the manifest that was signed and anchored; the bundle manifest itself is not public yet.

Licensing and credits. Explainer narration: AI voice (ElevenLabs); music: AI-generated (ElevenLabs). Apart from SDXL-Turbo itself, which is under the Stability AI Community License and was used here for non-commercial research, we used only MIT- or Apache-licensed material (Goodfire's SAE and loader, CLIP, OWLv2, Grounding DINO, SigLIP), and we will ask before using anything else.