Skip to content
HONESTY.TOOLSAn independent research observatory

Audit notes

We checked Will DePue’s checkpoint-forensics fingerprint

Published · Updated

We checked Will DePue’s “checkpoint forensics” report and announcement thread against two public DeepSeek checkpoints: deepseek-ai/DeepSeek-V3-Base and deepseek-ai/DeepSeek-V4-Flash. His reproduction scripts, row fixtures and test outputs are available to reviewers on request; we did not request them and did not use them. We used our own Philox emulator, written from public PyTorch and cuRAND source, and ran on CPU only.

The claim we checked is numerical: untrained embedding rows can keep the init seed plus the SM-count layout PyTorch used, so both can be recovered from public weights. Their claims stay theirs. The numbers below are what we independently reproduced on these two checkpoints. We used the V3-Base checkpoint linked in his sources, not the instruct checkpoint.

The untrained rows match. Bit-for-bit stays his claim. V4-Flash: 468 rows at std 0.0060, 461 and 445 rows at 148 SMs; V3-Base: all 489 rows at 132 SMs; ~8σ chance ceiling holds. Honesty.tools reading note · Will DePue, checkpoint forensics. Silent loop. Download the MP4.

The untrained rows are real

DeepSeek-V4-Flash has 468 rows at std 0.0060 with kurtosis 3.00, consistent with raw init — not the config’s 0.02. A 469th row, id 105044, sits just outside that tight cluster. His selected set of 469 is the 468 plus that row.

DeepSeek-V3-Base has 489 such rows, including all 465 unused token ids. His published 57,344 values (eight rows) are a conservative subset of that; we did not try to recover which eight he plotted.

Both configs list initializer_range: 0.02. The empirical init on these untrained rows is 0.006.

Seed and layout, independently reproduced

With his published seed 953049583 and starting offset 0, V4-Flash matches a 148-SM layout: the same 461 rows with r > 0.99 and 445 with r > 0.9999 that he reports. On a GPU he reports the 445 as bit-for-bit; on CPU we can show r > 0.9999. At 132, 152 or 108 SMs, or seed+1 or offset+1, r is about 0.

Condition overall r median r min r n with r > 0.99 n with r > 0.9999
148 SM 0.99970860 0.99999863 0.982807 461 445
132 SM −0.00033 −0.00048 −0.045 0 0
152 SM 0.00033 0.00072 −0.039 0 0
108 SM 0.00012 0.00065 −0.038 0 0
148 SM, seed +1 0.00039 0.00157 −0.041 0 0
148 SM, offset +1 −0.00055 −0.00111 −0.046 0 0

The seven tight-cluster rows with r < 0.99, plus id 105044 (r = 0.884 at 148 SMs), are consistent with his note that eight selected rows were partly changed by training.

Same seed, different layout for V3-Base. All 489 init-like rows match 132 SMs; none match 148. That’s about 61× the 57,344 values he reported.

| Condition | overall r | median r | min r | n with r > 0.99999 | |---|---:|---:|---:|---:|---:| | 132 SM, 465 unused | 0.99999863 | 0.99999863 | 0.99999855 | 465 / 465 | | 148 SM, 465 unused | 0.00063 | 0.00093 | −0.028 | 0 | | 132 SM, seed +1, 465 unused | 0.00091 | 0.00115 | −0.029 | 0 | | 132 SM, the extra 24 cluster rows | all r > 0.999999 | — | — | 24 / 24 | | 148 SM, the extra 24 | overall −0.0017 | — | — | 0 |

We do not claim bit-for-bit identity. The remaining 1−r gap of about 1.4×10⁻⁶ is the systematic difference between host math and CUDA device approximations. We did not run on a real GPU, so bit-for-bit stays his claim.

The chance ceiling holds up

His ~8σ chance ceiling over ~6×10¹² seed and layout combinations holds up: the expected maximum is about 7.3σ, and P(≥8σ) is about 0.4%. More precisely, for N = 6×10¹² i.i.d. standard normals, the EVT expected maximum is 7.2857 and P(max ≥ 8) is 0.00373. “About 8σ” is a conservative search-wide ceiling. We did not re-derive his 49–1,385σ figures for the six models.

What we did not check

Not checked: his other four models, the Moonshot seed, the 24/24 table, the 49–1,385σ figures, and the GPU-count estimates. Scripts, fixtures, and Philox-offset documentation remain “on request.” Offset 0 worked here, so we did not need them; they are still not public.

The fingerprint is of how the weights were initialized.

Sources and pinned receipt


Receipts: Ford’s RECEIPT_c24.md (sha256 8a91c15bce5e5d7220fc0bc3422c56455b1d59b39e9fdf497cdbbe0037ec3c71) for the C24 checkpoint-forensics pass (Nov 12, 2026, America/Denver). Independent Philox reproduction on two public DeepSeek checkpoints, without his scripts. CPU only; no paid compute. Verdict: untrained rows confirmed; seed 953049583 and offset 0 recover a 148-SM layout on V4-Flash (461 / 445 rows) and a 132-SM layout on V3-Base (all 489 rows); ~8σ chance ceiling consistent; bit-for-bit stays his claim. Ford interest: 9/10.

← All posts