# Changelog — versions, verdicts and corrections · Honesty.Tools

Source: https://www.honesty.tools/changelog

Static authored content from the public HTML. Interactive outputs are not simulated. Source-specific research, product and evidence limits remain in the text; this export is not new verification.

the record

# Changelog — what changed, and whether it changed verdicts.

## 2026-11-05 · Blog: Astra covert-underperformance reading note

A new [blog post](https://www.honesty.tools/blog/astra-covert-underperformance-reading-note/)  by Abe is a reading note on OpenAI’s Astra “covert underperformance” report. Without released task pairs we did not recheck the headline 10% vs 16%; the card’s own arithmetic held, and the ethical-doubt filter conditions on framing. Not a full audit. No verdict, number or certificate changed.

## 2026-11-05 · Blog: weight-space safety edit on Qwen3-VL-8B

A new [blog post](https://www.honesty.tools/blog/weight-space-safety-edit-qwen3-vl/)  by Abe checks a weight-space safety edit on Qwen3-VL-8B, rebuilt from Amrit Singh Bedi’s published editing log. In our rebuild, FigStep attacks that get through fall from 28.2% to 0.2%, while 54.4% of safe image-text requests are refused; utility stays flat. Its video and cards are drawn from those numbers. No verdict, number or certificate changed.

## 2026-11-03 · Blog: weight edits post, interim update

The [weight edits post](https://www.honesty.tools/blog/weight-edits-and-the-evidence-for-understanding/#update-do-the-models-read-the-weights)  has a new section on whether the models read the weights at all: the car fix in Experiment 4, a small audited check with and without the weights, our pre-registered study, and a leak we found in our own first batch. The clean rerun and planted-bug results are not in yet; we’ll add them to the post when they’re scored. Findings 1 to 5 are unchanged. No verdict, number or certificate changed.

## 2026-11-02 · Blog: weight edits and the evidence for understanding

A new [blog post](https://www.honesty.tools/blog/weight-edits-and-the-evidence-for-understanding/)  by Abe looks at MIT CSAIL’s weight-space programming work, where an AI edited a small model’s weights and lifted its MATH-500 score from 7.9% to 48.1%. We audited the released records: the gains hold up, the edit rebuilds exactly, some failures remain, and telling models apart is not yet explaining them. It also reanalyses our own exploratory Qwen3-4B probe pilot and says what this means for AFFERENT. No model was run again. Its charts are drawn from the released records and our pilot scores. No verdict, number or certificate changed.

## 2026-11-02 · Receipt log: how often we add checkpoints

[/evidence](https://www.honesty.tools/evidence#checkpoint-cadence)  now says how often we add receipt-log checkpoints: within 7 days of a receipt entering the log, with a “last checked” date updated at least monthly when nothing is added. None has been added since checkpoint 23 because nothing new has entered the log. It also says what this does not prove: a log we run ourselves cannot show that nothing was left out. [/research](https://www.honesty.tools/research#integrity)  and [KEYHOLDER.md](https://www.honesty.tools/verifier/log/KEYHOLDER.md)  point to it. No verdict, number or certificate changed.

## 2026-11-02 · Blog: how to check a probe monitor fairly

A new [blog post](https://www.honesty.tools/blog/checking-a-probe-monitor-fairly/)  by Abe sums up OpenAI’s post on the activation probes it runs on GPT-5.6 completions, credits it for publishing the method, and lists the questions a fair outside check would ask: selection and seeds, drift, shortcuts, harmless look-alikes, and numbers tied to runs. It says we are starting an independent check of the method on open-weight models; there are no results yet. Its three diagrams contain no data. No verdict, number or certificate changed.

## 2026-11-01 · Blog: what makes a fair comparison

A new [blog post](https://www.honesty.tools/blog/what-makes-a-fair-comparison/)  by Abe looks at Nikita Pavlichenko’s study of LLM design changes since GPT-3: one budget, one change at a time, stack the winners, limits stated first. It says we will report each model’s design for every honesty-geometry pair, and check AFFERENT’s model design before any lock. Its chart is redrawn from numbers in his post. No verdict, number or certificate changed.

## 2026-10-28 · Research: integrity section now mentions the witness co-signatures

[/research#integrity](https://www.honesty.tools/research#integrity)  still said independent witness co-signing was not yet in place. It now says checkpoint 23 is co-signed by rgdd.se/poc-witness and Markovian on the Witness Network testing list, which may be reset, and links the co-signatures, matching [/evidence](https://www.honesty.tools/evidence#open) . No verdict, number or certificate changed.

## 2026-10-28 · Blog: what a borderline pass means

A new [blog post](https://www.honesty.tools/blog/what-a-borderline-pass-means/)  by Abe walks through the [HLE batch 2 result](https://www.honesty.tools/epoch-crosswalk#batch2-v2)  in plain words: the borderline grouped test, the controls, the length caveat, D6 and D7, why batch 1 was voided rather than fixed, and the limits. Its charts are drawn from the published numbers. No verdict, number or certificate changed.

## 2026-10-28 · Blog: check our work without trusting us

A new [blog post](https://www.honesty.tools/blog/check-our-work-without-trusting-us/)  by Abe explains the receipt log in plain words: the append-only log, signed checkpoints, the Rekor timestamps and the two outside witnesses. It includes commands to check the log yourself, run against the live site before posting, and says what the log does not prove. Blog pages now style code blocks. No verdict, number or certificate changed.

## 2026-10-28 · Receipt log: witness co-signatures published

The co-signatures from rgdd.se/poc-witness and Markovian are now public in [/verifier/log/](https://www.honesty.tools/verifier/log/KEYHOLDER.md) : [cosigned/23](https://www.honesty.tools/verifier/log/cosigned/23)  (both witnesses), [cosigned/22](https://www.honesty.tools/verifier/log/cosigned/22)  (rgdd.se/poc-witness), their keys with links to each witness’s own page, and a small offline checker, [verify_cosignatures.py](https://www.honesty.tools/verifier/verify_cosignatures.py) . Both witnesses are still on the Witness Network testing list, which may be reset. [/evidence](https://www.honesty.tools/evidence#open)  no longer says the co-signatures are unpublished. No verdict, number or certificate changed.

## 2026-10-28 · Blog: notes on the Goodfire re-check

A new [blog post](https://www.honesty.tools/blog/what-we-learned-re-checking-a-goodfire-claim/)  by Abe, Kevin Russell’s AI assistant on our team, retells the [/goodfire](https://www.honesty.tools/goodfire)  work in plain words. Every blog page now has its own preview card, and blog headlines are calmer in size. No verdict, number or certificate changed.

Later the same day: the post now has pictures and a video, all reused from the work itself: the 30-second video Data made for the X thread, before-and-after images, the score chart, the face-size boxes, one rating pair and the bug card. Text trimmed slightly; no number or caveat changed.

## 2026-10-28 · Receipt log: two outside witnesses co-signed checkpoint 23

rgdd.se/poc-witness and Markovian, both on the Witness Network testing list, co-signed checkpoint 23. The testing list is meant for testing and may be reset. [/evidence](https://www.honesty.tools/evidence#open)  no longer says no witness co-signs our checkpoints. The co-signatures are not published yet. No verdict, number or certificate changed.

## 2026-10-28 · Receipt log: checkpoint 23 published

The [receipt log](https://www.honesty.tools/verifier/log/KEYHOLDER.md)  now has 23 entries. The new entry is the Goodfire run bundle (its hash only; the bundle stays private). Checkpoint 23 was signed and anchored in Rekor on 27 October. A [consistency proof](https://www.honesty.tools/verifier/log/proofs/consistency-22-23.json)  shows it only adds to checkpoint 22; the first 22 entries are unchanged. Each checkpoint is now kept at /verifier/log/checkpoints/<size>. No verdict, number or certificate changed.

## 2026-10-28 · HLE batch 2 result: PASS (grouped test borderline)

The revised batch-2 plan (v2) ran on HLE. Result: PASS. The grouped test passed, but only just (borderline; adjusted p 0.0466). The pair test passed (adjusted p 0.2962). Planted copies were all caught and both shuffle controls passed. Answer length is the weakest spot; this is descriptive only. Two plan changes (D6, D7) are published with their hashes and times. The score pack, its hashes and the plan are on [/epoch-crosswalk](https://www.honesty.tools/epoch-crosswalk#batch2-v2) . Batch 1 on HLE stays void. /audit and llms.txt now mention the batch-2 result.

Later the same day: wording tightened after an internal numbers audit; no number changed.

## 2026-10-28 · HLE batch 2: pilot script published; run started

On the [Epoch crosswalk](https://www.honesty.tools/epoch-crosswalk#batch2-v2)  page, the pilot script that holds the 75% rule and its summary are now downloads, with their sha256. A timing line explains the 15:25 label in the plan. The revised plan is now running on HLE; the result will be posted there, pass or fail. No verdict, number or certificate changed.

## 2026-10-28 · HLE batch 2: revised plan passed its check and is locked

On the [Epoch crosswalk](https://www.honesty.tools/epoch-crosswalk#batch2)  page, the revised batch-2 plan passed the same synthetic check and was locked at 06:15 MDT. It adds one cue level, at 75% of questions, and keeps every threshold. It has not been run on HLE yet. The gate file and the locked plan are published with their sha256. The failed first check stays on the page as it was. On [/goodfire](https://www.honesty.tools/goodfire#locked) , both locked studies now read “Started; stopped before a valid result”, with one line on why. No verdict, number or certificate changed.

## 2026-10-27 · HLE batch 2: pre-run check failed, not run

The [Epoch crosswalk](https://www.honesty.tools/epoch-crosswalk#batch2)  page now shows that the second Humanity’s Last Exam batch failed its check on synthetic data, so it was not run. One test found a planted cue in only 32 of 50 datasets; the rule needed 40. The gate file is published with its sha256. On [/goodfire](https://www.honesty.tools/goodfire#locked) , the two locked plans now read “Locked. Not yet run.” No verdict, number or certificate changed.

## 2026-10-27 · Calmer headline sizes

Page titles and section headings are smaller across the site, so they read calmly on laptops and phones. Body text keeps its size. Sizes only. No copy, number, verdict or certificate changed.

## 2026-10-27 · Legibility pass

Body text is larger on phones, header and footer links are larger, and text that ran off small screens on several pages now wraps. Sizes and layout only. No copy, number, verdict or certificate changed.

## 2026-10-27 · Aurora background

The site background is now a soft aurora drawn in CSS only, with no scripts, and the sky is darker behind the hero copy. No content changed.

## 2026-10-27 · Epoch AI crosswalk, batch 1

On [/epoch-crosswalk](https://www.honesty.tools/epoch-crosswalk)  we ran our [/audit](https://www.honesty.tools/audit)  cue test on benchmarks from Epoch AI’s Benchmark Reviews, under rules locked before any data was touched. GraphWalks: PASS, meaning no surface cue detected at this n. Humanity’s Last Exam: void, because the shuffled-label control fired. SimpleQA Verified: not testable this way. Epoch’s verdicts are shown as theirs and not re-scored. Not signed or anchored in Rekor. Independent of Epoch AI; they have not reviewed it.

## 2026-10-27 · Two Goodfire checks locked before running

Two more Goodfire checks, parameter decomposition and reward-hack labels, were written down and locked before any data was downloaded or anything was run. [/goodfire](https://www.honesty.tools/goodfire)  lists only the hash of each plan for now. The full texts will be published with the results, so anyone can check they match. No results yet.

## 2026-10-27 · Fix offered for Goodfire’s sae.py

While loading the SAE for the 7446 re-check we hit two small errors in sae.py in goodfire-ai/sdxl-turbo-interpretability, in forward and tiebreak. We sent the fix upstream as a [pull request](https://github.com/goodfire-ai/sdxl-turbo-interpretability/pull/1) . Neither error affects the encode / decode path behind our results.

## 2026-10-27 · Goodfire feature 7446 re-check, three-feature follow-up, anchored in Rekor

On [/goodfire](https://www.honesty.tools/goodfire) , an independent, pre-registered re-check of Goodfire’s SDXL-Turbo feature 7446 (“opening one’s mouth”). Yes, as tested: steering it reliably makes animals’ mouths look more open, and no random feature does. A pre-registered follow-up tested three more Goodfire features the same way and went 2 of 3: 4045 and 4071 PASS, 9820 FAIL. The run files were signed into our receipt log and anchored in Sigstore Rekor v2 (log2025-1, index 171750012). The log’s new leaf and the bundle are not public yet, and the work is not peer reviewed.

## 2026-10-24 · Site polish and a donor page preview

Every page now shares one header and footer. Page copy was tightened to match the evidence it cites, and the geometry findings now state their published numbers with links to the paper sections. A donor page, /donate, is up as a preview with sample data and no payment connected. No verdict, measurement or certificate changed.

## 2026-10-03 · Research lab, catalog and agent guide

Lab / Tools / Research / Evidence / Documentation / Collaborate navigation now connects 26 catalog instruments and 15 research families. The researcher-agent guide and machine-readable contracts share one catalog. Existing negative findings and corrections remain attached. This release adds no scientific measurement or independent replication. Privacy wording now distinguishes hosted API signup from browser calibration.

## Versions, verdicts and corrections

Changes to public tools, site interpretation and evidence stay dated here. Historical entries retain what changed and whether it changed a result. The October 3 site revision changes navigation and research descriptions; it reports no new experiment.

A trust instrument that updates itself owes you this page. Entries marked
verdict-changing  altered how reads classify — a read you ran before that date
may classify differently today. Wrong verdict? Tell us:
POST /api/feedback or hello@honesty.tools.
Every entry here is a receipt you can re-check yourself — download [aperture_verify.py](https://honesty.tools/verifier/aperture_verify.py)  and verify offline against the pinned key.

2026 · July 18

## Provenance you can check without us — the offline verifier stack.

The receipt stopped needing our server. Each item below is a dependency-light file a stranger
downloads and runs offline, checking our reads against a pinned key with no call back to us.
Provenance, not truth: these prove what was served and that a rule re-derives — never that the
answer is true.

- Open offline verifier — [aperture_verify.py](https://honesty.tools/verifier/aperture_verify.py) : one file, checks any receipt or certificate offline against the pinned key. No network, no account, no us.

- Transparency log — an RFC 6962 Merkle log (transparency_log.py): a suppressed or swapped receipt is third-party checkable via inclusion and consistency proofs. The log root is signed by our own key. On 28 October 2026, two independent witnesses on the Witness Network testing list, rgdd.se/poc-witness and Markovian, co-signed checkpoint 23.

- Flight recorder / session trails verdict-changing — verify a whole agent run offline (aperture_session.py, verify-session): every receipt's signature plus the Merkle root over the ordered trail. Tamper, reorder, or drop is caught.

- Deterministic per-answer witnesses — aperture.witness/v1 ([aperture_witness.py](https://honesty.tools/verifier/aperture_witness.py) , [WITNESS_SPEC](https://honesty.tools/verifier/WITNESS_SPEC.md) ): for fixed-rule answers, an exact witness a stranger re-derives offline — code re-executed in an attested sandbox, a math identity re-checked in SymPy, a registry fact proved present by a Merkle proof. A witness proves the rule fired and what was served, never that the answer is true; a grounded-registry witness proves a fact is in our registry, not true-in-world.

- /v1/verify is live — sign up, mint an sk-apt- key, POST to /v1/verify, get verdicts, metering, and a signed receipt. Billing is Stripe test-mode today, not yet real-money.

- Certificate epistemic trail — the certificate now shows the ordered trail of what settled a read, and every read has a [/read](https://www.honesty.tools/docs) receipt permalink you can re-open and re-verify.

2026 · June 30

## The reframe: honesty-tier, uniquely auditable, self-hostable.

The frontiers caught up on raw accuracy. So we stopped claiming a metric lead we no longer
hold and named what is actually defensible: a signed receipt you can verify yourself,
a model you can run on your own hardware, and the discipline to abstain rather than
bluff.

- What the receipts say. On a 147-claim battery, Photon answers 78.9% correct, is confidently-wrong on 2.7%, fabricates 0 of 36 designed traps, and abstains on 18% (96.7% correct when it does answer). The strongest frontier is more accurate at ~95%; same 2.7%. — WITHDRAWN 2026-07-26. Every figure here is measured on the n=147 battery, which our own null test [DISQUALIFIES](https://honesty.tools/verifier/batteries/verifydomain-claims-147.json) at lexicality A* 0.967: a classifier that never sees a model tells the fabricated claims from spelling, so none of these rates is readable as evidence. This page was not in the census that caught the others; the claim gate found it.

- The receipt is the moat verdict-changing — every read carries an ed25519 signature independently verifiable against the published key. The [evidence page](https://www.honesty.tools/evidence) now runs a real in-browser WebCrypto verify of stored reads; tampering breaks the signature. A verdict you can't re-check isn't evidence.

- Self-host shipped — Photon Base runs on your own hardware; the data never leaves. The honesty layer no longer requires trusting a remote service with your text.

- The activation gauge is a pre-filter, not the catch verdict-changing — the off-map / activation signal is a cheap research probe and attestation coordinate, used to triage. Fabrications are caught by grounding (verified offline registries) and cross-model checks, never by the activation reading alone.

- The Harness — the combined instrument is now named the Harness: a calibrated mid-brain that settles cheap claims locally, escalates the hard tail, and signs the result.

- /benchmarks retired — the old leaderboard framing invited a raw-metric race we don't claim to win. The honesty story lives on [/evidence](https://www.honesty.tools/evidence) now.

2026 · June 12

## The Model Notary: certificates now get checked daily.

- Daily drift canary over every aperture-verified registry model: a deterministic slice of its own calibration battery, re-scored with the stored probe and compared to the certificate (exact binomial, p<0.01). Latest result per model at [/api/notary](https://honesty.tools/api/notary) and as "verified <date>" / "⚠ DRIFT" chips on the [registry wall](https://www.honesty.tools/calibrate#registry) . Day one caught openai/gpt-4.1 unreachable on the calibration route — a vanished alias is a finding.

- Frontier liveness is now evidence-based across processes — /api/health reports live/down/unverified from recorded paid-call outcomes, file-backed.

- Citations and medical battery regimes (fabricated papers/books; fabricated drugs/conditions), ZIM-validated, registrable on /calibrate.

- High-stakes atlas gate verdict-changing — for medical/clinical queries, a confident read whose drug/condition is absent from the offline encyclopedia now downgrades to UNVERIFIED instead of shipping ON MAP (closed a live fabricated-drug miss: amlodipex, calcizan).

- aperture-gate 0.1.5 on PyPI — pip install aperture-gate.

- Battery results published — real drugs, conditions, and works pass clean; designed fabrications are caught or honestly abstained. A handful of fabricated paper titles rode a confident fingerprint to ON MAP, which became the next three fixes the same night:

- Citation-frame atlas gate verdict-changing — a query citing a quoted work title now gets the free offline-atlas title check on confident reads too; a missing title downgrades ON MAP → UNVERIFIED ("cannot confirm the work exists"), never to "fabrication" — most real papers have no encyclopedia article, and honest abstention is the right verdict for them. Post-gate re-run: no fabricated citation certifies as grounded; no real work is harmed.

- Web-rescue naming guard verdict-changing — a "real" web record must actually NAME the queried thing before it can rescue a flagged read. Found live: the fabricated drug trenadol was rescued to VERIFIED by search-engine autocorrect returning tramadol sources. Tightened after the regression run caught a flicker: for medical queries the name must appear in the sources themselves — the search answer echoes the queried name even when every source is about the autocorrected drug. The withheld rescue is disclosed in the payload.

- Fiction polarity at the record arbiter verdict-changing — "Is Sherlock Holmes a fictional character?" was ending "treat as fabrication": the record’s subject-is-fictional finding was read as answer-is-fabricated. When the model itself says the subject is fictional, the record agrees, and the atlas knows the entity, the read now confirms instead of refuting.

- Notary day one closed clean — 20/20 registry models verified, zero false positives, zero drift flags; transient-network false alarms eliminated via a control-call check (the gpt-4.1 “unreachable” read above resolved as exactly that once the check landed).

- Cold-deploy certificate strengthened — two never-analyzed strangers (a Mistral-lineage embedding fine-tune and phi-4) certified with zero hand-tuning; nulls dead with margin (shuffled-R 0.464 / 0.194). The [certificate page](https://www.honesty.tools/certificate) now cites these runs.

- The register-clear rule verdict-changing — "What is the capital of France?" was reading 89% off-map and renting a paid web rescue: the off-distribution pre-filter measures distance from news-register prose, so trivia questions about utterly real things crossed the gate. A flagged read whose subject is familiar (the discriminating gauge, same forward pass) and exactly charted in the offline atlas now clears at $0 — real trivia false-flags collapsed to none, with fake coverage held, re-proven on both batteries after deployment. The raw gauge value still ships in every payload, with the clear disclosed.

2026 · June 11

## Deep-audit remediation — the words instrument learned polarity.

- Words-reader polarity fix verdict-changing — correct answers about fiction and non-existence ("Sherlock Holmes is a fictional character", "there is no largest prime") no longer read OFF THE MAP. Real-answer false flags eliminated; detection traded down slightly to stay honest. Ported in lockstep to the SDK, browser, and CLI.

- The council never self-judges under the independence label — judge-unavailable degrades to UNCERTAIN with the fallback disclosed, never a flagship-graded "independent verification".

- Cert registration hardened — /v1 annotate consumes aperture-verified probes only; malformed probes rejected at the door.

- SDK 0.1.2 → 0.1.5 — ed25519 signature verification at use time (pinned registry key), expiry enforcement, streaming-time verdicts in gate.wrap(), the /v1/audit batch endpoint, smalltalk returns NON_CLAIM, dead gauges fail toward UNVERIFIED — never toward green.

- Evidence-page attribution corrected: the deep-read pipeline's catch and the activation pre-filter's own contribution are stated separately, never merged into a single headline figure.

2026 · June 10

## The fingerprint learned to read only what it calibrated on.

- Token-window fix verdict-changing — the fingerprint is scored on the certificate’s calibrated token budget instead of the whole answer; false positives on production-length real answers fell sharply with fake-detection held.

- Self-serve calibration (/calibrate, in-browser, key never leaves the tab) and the public registry wall, seeded first-party across ~20 frontier models; certificates ed25519-signed.

- Hedge lexicon +29 gate-tested refusal phrases verdict-changing — readers had been missing real refusals on some models (e.g. claude-sonnet’s "doesn’t appear in my knowledge base"); walls re-derived from saved answers.

2026 · June 8–9

## Public launch gate.

- Landing rebuilt (twin-catch hero); the Prism read and the edge gate restored after a stale-base regression; front-end libraries vendored, third-party CDNs dropped from the CSP.

- 49-agent fresh-eyes sweep before opening: ops-file leak quarantined, smalltalk no longer gets certificates, the papers published verbatim on [the evidence page](https://www.honesty.tools/evidence) .
