HONESTY.TOOLSAn independent research observatory
the record

Changelog — what changed, and whether it changed verdicts.

2026-11-05 · Blog: Astra covert-underperformance reading note

A new blog post by Abe is a reading note on OpenAI’s Astra “covert underperformance” report. Without released task pairs we did not recheck the headline 10% vs 16%; the card’s own arithmetic held, and the ethical-doubt filter conditions on framing. Not a full audit. No verdict, number or certificate changed.

2026-11-05 · Blog: weight-space safety edit on Qwen3-VL-8B

A new blog post by Abe checks a weight-space safety edit on Qwen3-VL-8B, rebuilt from Amrit Singh Bedi’s published editing log. In our rebuild, FigStep attacks that get through fall from 28.2% to 0.2%, while 54.4% of safe image-text requests are refused; utility stays flat. Its video and cards are drawn from those numbers. No verdict, number or certificate changed.

2026-11-03 · Blog: weight edits post, interim update

The weight edits post has a new section on whether the models read the weights at all: the car fix in Experiment 4, a small audited check with and without the weights, our pre-registered study, and a leak we found in our own first batch. The clean rerun and planted-bug results are not in yet; we’ll add them to the post when they’re scored. Findings 1 to 5 are unchanged. No verdict, number or certificate changed.

2026-11-02 · Blog: weight edits and the evidence for understanding

A new blog post by Abe looks at MIT CSAIL’s weight-space programming work, where an AI edited a small model’s weights and lifted its MATH-500 score from 7.9% to 48.1%. We audited the released records: the gains hold up, the edit rebuilds exactly, some failures remain, and telling models apart is not yet explaining them. It also reanalyses our own exploratory Qwen3-4B probe pilot and says what this means for AFFERENT. No model was run again. Its charts are drawn from the released records and our pilot scores. No verdict, number or certificate changed.

2026-11-02 · Receipt log: how often we add checkpoints

/evidence now says how often we add receipt-log checkpoints: within 7 days of a receipt entering the log, with a “last checked” date updated at least monthly when nothing is added. None has been added since checkpoint 23 because nothing new has entered the log. It also says what this does not prove: a log we run ourselves cannot show that nothing was left out. /research and KEYHOLDER.md point to it. No verdict, number or certificate changed.

2026-11-02 · Blog: how to check a probe monitor fairly

A new blog post by Abe sums up OpenAI’s post on the activation probes it runs on GPT-5.6 completions, credits it for publishing the method, and lists the questions a fair outside check would ask: selection and seeds, drift, shortcuts, harmless look-alikes, and numbers tied to runs. It says we are starting an independent check of the method on open-weight models; there are no results yet. Its three diagrams contain no data. No verdict, number or certificate changed.

2026-11-01 · Blog: what makes a fair comparison

A new blog post by Abe looks at Nikita Pavlichenko’s study of LLM design changes since GPT-3: one budget, one change at a time, stack the winners, limits stated first. It says we will report each model’s design for every honesty-geometry pair, and check AFFERENT’s model design before any lock. Its chart is redrawn from numbers in his post. No verdict, number or certificate changed.

2026-10-28 · Research: integrity section now mentions the witness co-signatures

/research#integrity still said independent witness co-signing was not yet in place. It now says checkpoint 23 is co-signed by rgdd.se/poc-witness and Markovian on the Witness Network testing list, which may be reset, and links the co-signatures, matching /evidence. No verdict, number or certificate changed.

2026-10-28 · Blog: what a borderline pass means

A new blog post by Abe walks through the HLE batch 2 result in plain words: the borderline grouped test, the controls, the length caveat, D6 and D7, why batch 1 was voided rather than fixed, and the limits. Its charts are drawn from the published numbers. No verdict, number or certificate changed.

2026-10-28 · Blog: check our work without trusting us

A new blog post by Abe explains the receipt log in plain words: the append-only log, signed checkpoints, the Rekor timestamps and the two outside witnesses. It includes commands to check the log yourself, run against the live site before posting, and says what the log does not prove. Blog pages now style code blocks. No verdict, number or certificate changed.

2026-10-28 · Receipt log: witness co-signatures published

The co-signatures from rgdd.se/poc-witness and Markovian are now public in /verifier/log/: cosigned/23 (both witnesses), cosigned/22 (rgdd.se/poc-witness), their keys with links to each witness’s own page, and a small offline checker, verify_cosignatures.py. Both witnesses are still on the Witness Network testing list, which may be reset. /evidence no longer says the co-signatures are unpublished. No verdict, number or certificate changed.

2026-10-28 · Blog: notes on the Goodfire re-check

A new blog post by Abe, Kevin Russell’s AI assistant on our team, retells the /goodfire work in plain words. Every blog page now has its own preview card, and blog headlines are calmer in size. No verdict, number or certificate changed.

Later the same day: the post now has pictures and a video, all reused from the work itself: the 30-second video Data made for the X thread, before-and-after images, the score chart, the face-size boxes, one rating pair and the bug card. Text trimmed slightly; no number or caveat changed.

2026-10-28 · Receipt log: two outside witnesses co-signed checkpoint 23

rgdd.se/poc-witness and Markovian, both on the Witness Network testing list, co-signed checkpoint 23. The testing list is meant for testing and may be reset. /evidence no longer says no witness co-signs our checkpoints. The co-signatures are not published yet. No verdict, number or certificate changed.

2026-10-28 · Receipt log: checkpoint 23 published

The receipt log now has 23 entries. The new entry is the Goodfire run bundle (its hash only; the bundle stays private). Checkpoint 23 was signed and anchored in Rekor on 27 October. A consistency proof shows it only adds to checkpoint 22; the first 22 entries are unchanged. Each checkpoint is now kept at /verifier/log/checkpoints/<size>. No verdict, number or certificate changed.

2026-10-28 · HLE batch 2 result: PASS (grouped test borderline)

The revised batch-2 plan (v2) ran on HLE. Result: PASS. The grouped test passed, but only just (borderline; adjusted p 0.0466). The pair test passed (adjusted p 0.2962). Planted copies were all caught and both shuffle controls passed. Answer length is the weakest spot; this is descriptive only. Two plan changes (D6, D7) are published with their hashes and times. The score pack, its hashes and the plan are on /epoch-crosswalk. Batch 1 on HLE stays void. /audit and llms.txt now mention the batch-2 result.

Later the same day: wording tightened after an internal numbers audit; no number changed.

2026-10-28 · HLE batch 2: pilot script published; run started

On the Epoch crosswalk page, the pilot script that holds the 75% rule and its summary are now downloads, with their sha256. A timing line explains the 15:25 label in the plan. The revised plan is now running on HLE; the result will be posted there, pass or fail. No verdict, number or certificate changed.

2026-10-28 · HLE batch 2: revised plan passed its check and is locked

On the Epoch crosswalk page, the revised batch-2 plan passed the same synthetic check and was locked at 06:15 MDT. It adds one cue level, at 75% of questions, and keeps every threshold. It has not been run on HLE yet. The gate file and the locked plan are published with their sha256. The failed first check stays on the page as it was. On /goodfire, both locked studies now read “Started; stopped before a valid result”, with one line on why. No verdict, number or certificate changed.

2026-10-27 · HLE batch 2: pre-run check failed, not run

The Epoch crosswalk page now shows that the second Humanity’s Last Exam batch failed its check on synthetic data, so it was not run. One test found a planted cue in only 32 of 50 datasets; the rule needed 40. The gate file is published with its sha256. On /goodfire, the two locked plans now read “Locked. Not yet run.” No verdict, number or certificate changed.

2026-10-27 · Calmer headline sizes

Page titles and section headings are smaller across the site, so they read calmly on laptops and phones. Body text keeps its size. Sizes only. No copy, number, verdict or certificate changed.

2026-10-27 · Legibility pass

Body text is larger on phones, header and footer links are larger, and text that ran off small screens on several pages now wraps. Sizes and layout only. No copy, number, verdict or certificate changed.

2026-10-27 · Aurora background

The site background is now a soft aurora drawn in CSS only, with no scripts, and the sky is darker behind the hero copy. No content changed.

2026-10-27 · Epoch AI crosswalk, batch 1

On /epoch-crosswalk we ran our /audit cue test on benchmarks from Epoch AI’s Benchmark Reviews, under rules locked before any data was touched. GraphWalks: PASS, meaning no surface cue detected at this n. Humanity’s Last Exam: void, because the shuffled-label control fired. SimpleQA Verified: not testable this way. Epoch’s verdicts are shown as theirs and not re-scored. Not signed or anchored in Rekor. Independent of Epoch AI; they have not reviewed it.

2026-10-27 · Two Goodfire checks locked before running

Two more Goodfire checks, parameter decomposition and reward-hack labels, were written down and locked before any data was downloaded or anything was run. /goodfire lists only the hash of each plan for now. The full texts will be published with the results, so anyone can check they match. No results yet.

2026-10-27 · Fix offered for Goodfire’s sae.py

While loading the SAE for the 7446 re-check we hit two small errors in sae.py in goodfire-ai/sdxl-turbo-interpretability, in forward and tiebreak. We sent the fix upstream as a pull request. Neither error affects the encode / decode path behind our results.

2026-10-27 · Goodfire feature 7446 re-check, three-feature follow-up, anchored in Rekor

On /goodfire, an independent, pre-registered re-check of Goodfire’s SDXL-Turbo feature 7446 (“opening one’s mouth”). Yes, as tested: steering it reliably makes animals’ mouths look more open, and no random feature does. A pre-registered follow-up tested three more Goodfire features the same way and went 2 of 3: 4045 and 4071 PASS, 9820 FAIL. The run files were signed into our receipt log and anchored in Sigstore Rekor v2 (log2025-1, index 171750012). The log’s new leaf and the bundle are not public yet, and the work is not peer reviewed.

2026-10-24 · Site polish and a donor page preview

Every page now shares one header and footer. Page copy was tightened to match the evidence it cites, and the geometry findings now state their published numbers with links to the paper sections. A donor page, /donate, is up as a preview with sample data and no payment connected. No verdict, measurement or certificate changed.

2026-10-03 · Research lab, catalog and agent guide

Lab / Tools / Research / Evidence / Documentation / Collaborate navigation now connects 26 catalog instruments and 15 research families. The researcher-agent guide and machine-readable contracts share one catalog. Existing negative findings and corrections remain attached. This release adds no scientific measurement or independent replication. Privacy wording now distinguishes hosted API signup from browser calibration.

Versions, verdicts and corrections

Changes to public tools, site interpretation and evidence stay dated here. Historical entries retain what changed and whether it changed a result. The October 3 site revision changes navigation and research descriptions; it reports no new experiment.

A trust instrument that updates itself owes you this page. Entries marked verdict-changing altered how reads classify — a read you ran before that date may classify differently today. Wrong verdict? Tell us: POST /api/feedback or hello@honesty.tools. Every entry here is a receipt you can re-check yourself — download aperture_verify.py and verify offline against the pinned key.
2026 · July 18

Provenance you can check without us — the offline verifier stack.

The receipt stopped needing our server. Each item below is a dependency-light file a stranger downloads and runs offline, checking our reads against a pinned key with no call back to us. Provenance, not truth: these prove what was served and that a rule re-derives — never that the answer is true.

2026 · June 30

The reframe: honesty-tier, uniquely auditable, self-hostable.

The frontiers caught up on raw accuracy. So we stopped claiming a metric lead we no longer hold and named what is actually defensible: a signed receipt you can verify yourself, a model you can run on your own hardware, and the discipline to abstain rather than bluff.

2026 · June 12

The Model Notary: certificates now get checked daily.

2026 · June 11

Deep-audit remediation — the words instrument learned polarity.

2026 · June 10

The fingerprint learned to read only what it calibrated on.

2026 · June 8–9

Public launch gate.