Honesty.tools leaderboard — source attribution

This page reports attributed numeric measurements and original analysis. It is
not an official leaderboard of the source authors, and implies no endorsement.
Observation records retain source URLs, configurations, dates and limitations.
We do not redistribute source papers, raw response corpora, datasets or source
website snapshots. Scores are frozen observations, not live model evaluations.

Honesty.tools evidence benchmark: https://www.honesty.tools/benchmark/
Complete first-pass development measurements; low effort and dependent phases.

Vectara Hallucination Leaderboard:
https://github.com/vectara/hallucination-leaderboard
Snapshot af0e3207741054b44e7c87688a40a815ecf446df; Apache-2.0 (VECTARA_LICENSE.txt).
Adaptation: native fields parsed, source-scoped ranks and profiles added; no
source-reported performance values changed. Source license notice is retained.

HAL Agent Reliability — SAgE / Princeton University:
https://hal.cs.princeton.edu/reliability/
https://hal.cs.princeton.edu/reliability/methodology/
Attributed aggregate measurements and source standard errors, accessed 2026-10-11.
Original website and benchmark corpora are not distributed with this page.

ALCE — Gao, Yen, Yu and Chen (2023), Enabling Large Language Models to Generate
Text with Citations: https://arxiv.org/abs/2305.14627
https://github.com/princeton-nlp/ALCE (source repository notice: ALCE_LICENSE.txt).
Selected Table 18 ASQA VANILLA measurements and three-run standard deviations;
Table 7/18 discrepancy remains disclosed. The paper and benchmark are not copied.

BeHonest — BeHonest authors / GAIR (2024):
https://github.com/GAIR-NLP/BeHonest
Only aggregate cue-change rates independently reconstructed from published
retained judge outputs are distributed. Original response text, code and dataset
are not distributed. No blanket license to those source materials is asserted.

Definitions, diagnostic notes and evaluation plans are Honesty.tools analyses.
Missing coverage is not failure. Descriptive ranks do not prove statistical
superiority, general honesty, intent, alignment or certification.

SimpleQA — historical OpenAI provider-reported results:
https://github.com/openai/simple-evals
21 first-party result rows from the pinned table, deprecated July 2025.
Copyright (c) 2024 OpenAI; MIT (SIMPLEQA_LICENSE.txt). Third-party rows excluded.
Adaptation: only numeric factual-accuracy values imported, with source/configuration
profiles, descriptive ranks and original diagnostic notes. No trial counts or
abstention rates are inferred. Source commit and digest accompany every value.

FaithEval — Yifei Ming et al. / Salesforce AI Research (2024):
https://github.com/SalesforceAIResearch/FaithEval
Copyright (c) 2024 Salesforce, Inc.; Apache-2.0 (FAITHEVAL_LICENSE.txt).
Adaptation: 108 labeled measurements transcribed from three pinned author charts;
all labels checked at original resolution. Source images and datasets are not
redistributed. Baseline and task-conditioned results remain separate. Following
counterfactual context is not world-truth accuracy. No confidence interval inferred.

Honesty.tools trajectory analysis: evidence/trajectory.json
170 additional aggregates derived from already retained first-pass responses;
no additional trials. Question selection, explicit ledger update/stability and
delivery use declared opportunity denominators. Selected-path metrics are unranked.

TruthfulQA — Stephanie Lin, Jacob Hilton and Owain Evans (2021):
https://github.com/sylinrl/TruthfulQA
Apache-2.0 (TRUTHFULQA_LICENSE.txt).
Adaptation: first three metric columns from the complete four-model v0 baseline
table; human truthfulness and informativeness retained separately. GPT-judge
values are evaluator context, not independent replications. Non-answers can be
truthful; marginal truth/info rates do not identify their intersection. Default
QA and no-prefix prompting remain separate cohorts. Current binary tasks and
updated-reference results are excluded. Source commits/digests accompany values.

Honesty.tools family diagnostics: evidence/families.json
Aggregate drilldowns over the retained first pass, with correct decisions,
valid incorrect decisions and unusable outcomes partitioned separately. Exact
conflict certificates and decision correctness are distinct. No raw responses,
case prompts or private scenario contents are published. No additional samples.

Epoch AI, Capabilities & benchmarking: SimpleQA Verified
https://epoch.ai/benchmarks/simple-qa-verified
https://epoch.ai/benchmarks/use-this-data
Data: https://epoch.ai/data/benchmark_data.zip (accessed 2026-10-13).
CC BY 4.0: https://creativecommons.org/licenses/by/4.0/
The original data notice is retained in EPOCH_DATA_NOTICE.txt.
Adaptation: 90 source model/configuration records with accuracy and source standard
error converted from fractions to percentages. Original decimals, labels, source
run IDs and dates remain in evidence/epoch-simpleqa-verified.json. No source values
are selected by performance. No questions, model outputs or execution logs copied.
Per-run prompt, judge, provider and scored denominator are unverified. This is a
separate factual-accuracy source, not original SimpleQA or whole-response honesty.
All records are unranked context and contribute no weight to the frozen Mark.

Honesty.tools published-policy development assessment — 11 October 2026:
evidence/declared-20261011.json and evidence/families-declared.json
A complete separate acquisition over 128 scenarios and ten exact configurations.
The policy and assessment declaration were published and observed before the
first request, as recorded in root-held publication and launch receipts. This is
an operational chronology record, not a trusted timestamp attestation. All
scheduled opportunities remain in denominators, including unusable deliveries.
Scores use thirteen ratios and the declared dimension weights. Descriptive
ordering uses the displayed score; ties share rank. Fresh numerical instances
come from exposed development families, not an independently held-out study.
Earlier banks remain separately identified; no partial stopped run is pooled.

Honesty Mark cross-benchmark research prototype, policy v0.1:
honesty-mark-broad-policy-v0.1.json, evidence/broad-mark.json,
evidence/broad-coverage.json and evidence/broad-profiles/*.json
A retrospective fixed-weight policy across six dimensions and ten instrument
slots. The provisional observed-evidence summary requires at least two contributing
source families and is conditional on its visible basket. Unmeasured slots retain
their full weight in possible full-score bounds; they are not observed zeros.
No current model has all required evidence for the full Mark. Source-reported
labels are reviewed associations, not verified identical weights or compute.
HAL agent results remain contextual; the BeHonest change heuristic is not inverted
into an honesty score. Complementary measures and repeated tables receive no extra
vote. Policy utility transforms and marginal bounds are not source-author scores
or statistical confidence intervals. There are no cross-basket or lab honesty ranks.
Model-specific evidence plans: evidence/broad-evidence-plans.json
Plans describe the ten fixed instrument slots as point, bounded or missing
evidence and bind the current policy and model-label associations. A downloaded
request is preparation for review, not an accepted submission or certification.
Fresh measurements and validated instruments require a successor application;
they never silently replace frozen evidence.

The broad application pins the retained first bank independently of any newer
Evidence Behavior assessment. Its model profile exports preserve exact old hashes.

Comparison context: evidence/broad-comparison-context.json
The unchanged frozen aggregate is accompanied by exact source-weight shares and
recorded configuration comparisons. Shared scoring baskets do not establish
matched execution. Unknown fields remain unknown; source counts and weights do
not establish independence or effective sample size. Scores and weights are unchanged.

Evidence Behavior research index, policy v0.2:
honesty-mark-policy-v0.2.json and evidence/honesty-index.json
Thirteen declared success ratios form three equally weighted dimensions.
Component ratios and weighted score gaps remain traceable to the corresponding
verified evidence bank. Overlapping components are not independent samples.
This is a scoped aggregate across the mathematical evidence benchmark modules,
not all imported external benchmarks, universal honesty, alignment or certification.
Retrospective scores remain exploratory and alphabetically presented. Prospective
assessment requires a separately published policy/plan declaration before calls.
Policy-weight sensitivity is not a statistical confidence interval. Other sources
retain their native scores and exact configurations; missing evidence is not zero.

Honesty.tools Evidence under pressure development kit v0.1:
tools/pressure-lab-v0.1.zip and tools/pressure-lab-v0.1.md
Original public-safe synthetic instrument and software tests; MIT license scoped
to the kit, included in its archive. Deterministic simulations are software
controls, not model measurements or empirical construct validation. This kit
is not admitted to the Honesty Mark. No private benchmark bank is distributed.

Honesty.tools first evidence-pressure development pilot:
tools/pressure-pilot-v0.1.json and tools/pressure-pilot-v0.1.md
Two exact model/provider/configuration results on 32 shared exposed development
bases, with nine dependent continuations per usable initial. The local plan was
frozen before calls, not publicly preregistered or independently timestamped.
Every declared arm and paired contrast is retained. Results assess this candidate
instrument and do not change any Honesty Mark component, source score or rank.
Only aggregate results and public-safe reproduction metadata are exported.

Honesty.tools selective-trust development pilot:
tools/selective-trust-pilot-v0.1.json and tools/selective-trust-pilot-v0.1.md
Two model/provider/configuration results on 24 related synthetic registry cases,
with ten dependent continuations per usable initial. A stipulated source policy
defines which additive records count; this does not authenticate real sources.
Development followed the earlier pressure pilot, with no selective-trust outcomes
before the local freeze. No held-out or human-validation claim. All comparisons
remain descriptive and do not change the Honesty Mark, existing scores or ranks.

Honesty.tools measurement audit: evidence/measurement-audit.json
Decision correctness, incorrectness and unusable delivery partition all scheduled
phase outcomes for each named assessment. These are dependent observations, not
additional trials or an alternative score. Semantic flags overlap; token usage
is provider-reported accounting, not verified internal reasoning. Conditional
usable-response accuracy is diagnostic and does not replace scheduled denominators.
The published index weights and scores are unchanged. Model-specific caps and
routes remain visible; these results do not establish equal compute or every
model's maximum capability. Magnitude slices are descriptive, not causal effects.

Richard Ren et al., The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems (2025), arXiv:2503.03750v1. CC BY 4.0, https://creativecommons.org/licenses/by/4.0/. Versioned paper aggregates extracted; charts transcribed where applicable. No question corpus or current website leaderboard reused. Historical paper full-set results, not the current private-split leaderboard. Source explicit-honest and lie categories use elicited response consistency as a belief proxy; no direct proof of intentions. Evasion/no belief is separate from explicit honesty. The 1-minus-lie honesty score is not imported. @10 is a dependent repeated-pressure condition, not ten independent votes. Missing @10 values stay missing. No overall ranking or Mark admission.

Declan Jackson, William Keating, George Cameron and Micah Hill-Smith, AA-Omniscience (2025), arXiv:2511.13029v1. CC BY 4.0, https://creativecommons.org/licenses/by/4.0/. Versioned paper aggregates extracted; charts transcribed where applicable. No question corpus or current website leaderboard reused. Historical paper chart labels, not the current website leaderboard. Whole-number rounding retained. Index rewards correct answers and penalizes incorrect answers; always abstaining scores zero. Hallucination rate is incorrect/(partial+incorrect+not attempted), not incorrect/all or incorrect/attempted. Single-pass author-judged results, no calibration confidence claims, no Mark admission.

Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri and Samuel J. Bell, AbstentionBench (2025), arXiv:2506.09038v1. CC BY 4.0, https://creativecommons.org/licenses/by/4.0/. Versioned paper aggregates extracted; charts transcribed where applicable. No question corpus or current website leaderboard reused. Historical paper Table4 averages, not the noncommercial repository result CSV. Accuracy on answerable questions and abstention recall on should-abstain questions have different populations. Source excludes invalid correctness-judge outputs; no pooled counts or joint utility reconstructed. Average weighting and per-model realized counts are not recoverable from this table. No overall ranking or Mark admission.

Belief-R: Bryan Wilie, Samuel Cahyawijaya, Etsuko Ishii, Junxian He and Pascale Fung, Belief Revision: The Adaptability of Large Language Models Reasoning, arXiv:2406.19764v2 (2024-10-17), https://arxiv.org/html/2406.19764v2. Table A1 excerpt and derived factual records are attributed adaptations under CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/. Changes: rows expanded across prompting methods; published percentages retained. ConfidenceBench: Matthew ffrench-Constant, Daniel Yang, Xinmeng Huang and Sanyam Kapoor, arXiv:2607.20526v1 (2026-07-10), https://arxiv.org/html/2607.20526v1. Only attributed factual measurements and independently written metadata are exported. The article uses arXiv perpetual non-exclusive distribution permission; it is not treated as a general reuse license. No full article, figures, creative text, prompt template or private questions are redistributed. AGM-Bench: Ben Jenkins, official ICLR 2026 event https://iclr.cc/virtual/2026/10017503; numerical import blocked by inaccessible primary PDF. No model values fabricated. All imports are source-scoped context, not new execution, ranking or Mark admission.

Original FACTS Grounding: Google DeepMind / Google Research, December 17 2024, Table 6, https://storage.googleapis.com/deepmind-media/FACTS/FACTS_grounding_paper.pdf. Source paper copyright 2024 Google, all rights reserved. Only attributed numerical facts and original scope commentary are exported; no paper, figure, questions or answers. Original Grounding remains separate from Grounding v2. The 63 cells cover nine model labels, not 63 independent configurations or replications. Native 95% interval half-widths remain source-reported. No Mark admission or ranking.
The earlier bloom-import-status.json records the initial blocked retrieval. It is retained as historical evidence and superseded by the recovered primary figure below.

Bloom — Gupta et al., Anthropic, December 19 2025, https://alignment.anthropic.com/2025/bloom-auto-evals/ Figure 1. All 68 printed means across 17 labels and four distinct behavior configurations are retained. Only delusional sycophancy is mapped to honesty pressure context; instructed sabotage, self-preservation and self-preference remain behavioral context. The article says 16 models while its figure/sweeps contain 17; this discrepancy is preserved. Native two-decimal fractions are not independent trial counts; source SD bars are not numerically reconstructed. MIT covers retained repository code/configuration, not permission to redistribute article figures. Only attributed numerical facts and scope notes are exported; no images, HTML, prompts, transcripts or question banks. No current-model identity equivalence, native execution verification, ranking or Mark admission is claimed.

Retained repository configuration notice:
MIT License

Copyright (c) 2025 Safety Research

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Honesty Mark expanded evidence edition v0.2:
honesty-mark-broad-policy-v0.2.json, evidence/broad-mark-v0.2.json,
evidence/broad-evidence-plans-v0.2.json and evidence/broad-comparison-context-v0.2.json
Additive retrospective source-label edition: nine expressly bound Epoch SimpleQA
Verified correct-answer percentages and three MASK joint-event intervals. Epoch
native effort/configurations do not measure our own low-effort runs; per-run prompt,
provider, judge and realized denominators remain unverified. The native source
profiles keep their original context/admission flags; policy admission is confined
to this separately versioned application. MASK intervals use source common-population
assumptions and an explicitly unverified nearest-0.1 rounding convention. No midpoint,
joint point, confidence interval, universal honesty score, ranking or certification
is inferred. All six dimension weights and all 1,790 coordinates are retained.
The original v0.1 policy, application, plans, comparison context and profiles remain
available unchanged. The observatory's successor field is a labelled UI projection;
the complete source bindings, observations and coordinate audit are in the v0.2 export.
No questions, raw model responses, secrets or private repository code are added.

Honesty Mark expanded model coverage v0.3:
honesty-mark-broad-policy-v0.3.json and evidence/*-v0.3.json add 26
publisher-native Epoch SimpleQA Verified correct-answer statistics. Matching
retains literal model/version/date and native effort labels. After explicit
eligibility, run selection uses publisher start date then run ID, never score.
Selection is retrospective, not blinded, preregistered or a best-capability claim.
The matching audit accounts for all 179 entities and all 90 native runs, including
eligible alternatives and exclusions. Native prompts, providers, judges and
realised denominators remain unverified. Source profiles preserve their original
context flags; aggregate admission is limited to the separately versioned policy.
No current-ten proxy substitution, pressure midpoint, full Mark, global ranking,
certification, new model call or new expenditure is introduced. All earlier
edition exports and observatory.json remain byte-identical. The additional view
file is a display projection; the full application and audit are separate exports.
Epoch AI data attribution and source notices from the preserved package apply.
No raw questions, model responses, credentials or private code are included.

Honesty Mark native grounding and contextual revision v0.4:
This separately versioned policy adds five original FACTS Grounding combined
three-judge average statistics and five Belief-R direct-prompting BREU statistics.
It fills only missing coordinates; all previous measurements, weights and edition
exports are preserved. These are distinct native instrument variants, not a
calibrated common protocol, independent replications or proof of intentions.
FACTS: Google DeepMind, FACTS Grounding, original December 2024, Table 6.
https://storage.googleapis.com/deepmind-media/FACTS/FACTS_grounding_paper.pdf
Attributed numerical facts and original commentary only. No paper, figures,
questions or responses are redistributed. The source eligibility gate disqualifies
only when all three model judges reject. Printed 95% uncertainty is retained
separately; nominal question counts are not reconstructed realized denominators.
Belief-R: Bryan Wilie, Samuel Cahyawijaya, Etsuko Ishii, Junxian He and Pascale Fung,
Belief Revision: The Adaptability of Large Language Models Reasoning,
arXiv:2406.19764v2 (2024), Appendix C.2 Table A1.
https://arxiv.org/html/2406.19764v2#A3.T1
Source and attributed factual-table adaptations: CC-BY-SA-4.0.
https://creativecommons.org/licenses/by-sa/4.0/
Our direct-prompting (DP) condition is a retrospective declared choice, not the
authors' unique baseline or the best-performing method. Published BREU equally
weights marginal update/retain accuracy under commonsense suppression labels;
it is not a paired correct-initial-to-correct-update rate or universal deductive
revision accuracy. All methods remain in the matching audit. The scoring
population and handling of nonconforming outputs are not reconstructed.
Model labels are explicitly associated to existing bare-model source anchors.
Undated names do not establish dated revisions, provider or execution identity;
scaffold-only identities are excluded uniformly. Current-ten results are unchanged.
No full Mark, global rank, certification, new model call or expenditure is created.
All original source observations, archived editions and observatory.json remain
byte-identical. v0.4 is fetched initially; v0.3 is fetched when selected.
No raw questions, model responses, credentials or private code are included.