# HLE batch 2: NOT RUN (the synthetic check failed)

**HLE (multiple-choice, text-only subset): batch 2.** NOT RUN. The method did not pass its own pre-registered check on synthetic data, so it was not applied to HLE. No batch-2 result on HLE exists. The batch-1 HLE result stays VOID.

## What was planned
- Batch 1 on HLE was VOID because its control failed.
- Batch 2 was planned as a re-analysis of the same 467 questions (3,164 answer options). It would compare each question's keyed answer only with the wrong answers to that same question. It was not going to be a replication.
- There were two co-primary tests at alpha 0.025 each:
  - the **all-options test**, which compares the keyed answer with every wrong answer;
  - the **keyed-versus-one-wrong test**, which compares the keyed answer with one randomly drawn wrong answer.
- The approved design (sha256 `947218445023121a4e1080a0fa5a948cd306a495adf6e7fc93d012b91ffc9765`) set a rule in advance. Before any HLE data was touched, both tests had to pass a check on synthetic data that copies the batch-1 problem, with fixed pass marks.
- The pre-registration would have been locked only after that check passed. If the check failed, batch 2 would be NOT RUN, with no loosening.

## What the synthetic check found
The check ran on the DGX on Oct 27, 2026, from 08:04 to 13:56 MDT. It used no HLE data.

| Check | Pass mark | Result | |
|---|---|---|---|
| Batch-1 failure reproduced in synthetic data (private-word share 0.45) | at least 25 of 50 | 21 of 50 | not reproduced, so the next level was tried |
| Batch-1 failure reproduced (private-word share 0.60) | at least 25 of 50 | **25 of 50** | reproduced, so 0.60 was used |
| No-cue datasets, false alarms (share 0.60) | at most 9 of 200 per test | all-options 0, one-wrong 5 | pass |
| No-cue datasets, false alarms (share 0.75) | at most 9 of 200 per test | all-options 3, one-wrong 5 | pass |
| Cue planted, then labels shuffled, false alarms (share 0.60) | at most 9 of 200 per test | all-options 1, one-wrong 6 | pass |
| Cue planted, then labels shuffled, false alarms (share 0.75) | at most 9 of 200 per test | all-options 7, one-wrong 7 | pass |
| Planted one-character cue found, cue in 30% of questions | at least 40 of 50 per test | all-options 3, one-wrong 1 | miss, so 60% was tried |
| Planted one-character cue found, cue in 60% of questions | at least 40 of 50 per test | all-options 49, **one-wrong 32** | **FAIL** |

- **The false-alarm checks passed.** On synthetic data with the batch-1 problem built in and no real cue, neither test flagged a cue more often than the limit allowed.
- **The power check failed.** The keyed-versus-one-wrong test found a planted one-character cue in only 32 of 50 synthetic datasets with the cue in 60% of questions. It needed 40.
- The pre-registered rule gives no further step after the 60% level. So the outcome is NOT RUN: nothing was run on HLE or scored, nothing was locked, and no threshold was changed.

## Record
- Gate summary: `B2_SIM_GATE.json`, sha256 `bca24958c511620a060ceeafa7abf43c29b3a82cfb7f3033b6631581f5bc7c2d`.
- The unlocked pre-registration text is kept unchanged as the failed record: `PREREG_HLE_B2.md`, sha256 `4c34a85d07c92f062483bb03e2e08d30b97445463fb02ad2da644b830faf7df0`.