# Diagnostic report: gpt-4o-2024-05-13

Profile: hm-bfa2305d04775dc9691a
Profile digest: b10c11c4c9da69a385ac5e838f2ca5c8675f8fc32c3aa1c5bbfea5d6c3aa1f6a

Deterministic source-specific review prompts; no causal diagnosis or severity ranking.
Review items, not independent failures or summed events. Metrics and phases may overlap.

## Review items

### Short-form factual accuracy

Cohort: simpleqa:652c89d0ca9df547706735883097e9537d40dc47:provider-reported:assistant
Native score: 39.00%
Definition: Source-reported percentage graded correct on SimpleQA. This table does not provide incorrect or not-attempted rates; 100 minus accuracy is not a hallucination rate.

Some answers did not receive correct-answer credit under the provider-reported SimpleQA grading. The table does not separate incorrect answers from non-attempts.

Follow-up experiment: Reproduce the stated model and prompt setting on fresh fact questions. Report correct, incorrect and not-attempted outcomes together; validate the grader before interpreting factual error as overclaiming.

Scope: Historical OpenAI-reported results, not an independent Honesty.tools rerun. Prompt labels form separate cohorts. Per-row judged sample counts, repeats, uncertainty and abstention breakdown unavailable; no count or interval is inferred from rounded percentages.
Source: https://github.com/openai/simple-evals/blob/652c89d0ca9df547706735883097e9537d40dc47/README.md#benchmark-results

## Missing evaluation coverage

Honesty: belief_revision, citation_integrity, evidence_support, pressure_resistance, uncertainty
Alignment: authorized_goal_following, constraint_compliance, non_deception, oversight_and_correction, robustness

Missing coverage is not evidence of failure. Presence is not complete dimension assessment.

## Unmeasured metrics

None among the diagnostic rules.

## Reported endpoints

Endpoints are rounded source observations, not certification.

None among the diagnostic rules.

## Evaluation use

Use public development examples for diagnosis; reserve independent held-out tasks for evaluating any change. These suggestions are experiments, not proven training remedies.

No Honesty Mark awarded. This report digest does not authenticate an issuer.
Report digest: 7f9e0fde3869231ce0e9e1f8a067de30b2e54af6eb7265c4746dcf26074e3100