# Evidence for "I said agents need a change log. In my test it made them worse."

Public, redacted copies of the experiment behind the piece. The language model's name, the way it was reached,
costs, file paths and machine names are removed. Every other figure is as recorded.

| file | what it is |
|---|---|
| preregistration.md | The plan sealed at 05:09 UTC on 25 October 2026, before any agent episode: hypotheses, environment, arms, metrics, pass rules, and the changes made after sealing. |
| episodes.csv | One row per episode, 240 rows: 4 arms x 6 families x 10 seeds. |
| blocks.csv | One row per turn the arm D controller blocked, 53 rows, with the reason category. |
| diagnosis.csv | One row per fired fault per diagnoser and record shown. |
| summary.csv | The rates, paired differences, intervals and p values the piece quotes, calculated from the episode records using analysis code that is not included here. Its random generator, resampling order and quantile conventions are not documented, so a different bootstrap implementation can move an interval endpoint by a point or two. |

## episodes.csv columns

- arm: A notes, B notes plus a log the agent never sees, C a belief store the agent keeps and cites, D C plus a
  controller that blocks actions resting on a stale belief.
- family_no, family: 1 stale_door, 2 wrong_start, 3 delayed_update, 4 noise, 5 unobservable, 6 double_fault.
  "Under change" means families 1, 2, 3, 5 and 6.
- seed: 0 to 9. The same family and seed is the same house in every arm.
- success, false_completion: 1 or 0, scored by the simulator.
- stale_premise_actions: executed actions that only make sense in the world as the false premise has it.
- avoidable_stale_premise_actions: the same, counting only those taken after the evidence had reached the agent.
  This is the primary stale-premise metric.
- turns_used: out of 15.
- blocked_turns: arm D only, turns the controller blocked.
- repicked_wrong_key_after_correction: families 2 and 6 only. 1 if the agent picked up the wrong key after the
  supervisor's correction message had arrived.
- store_reverted_to_wrong_tag: arms C and D, families 2 and 6 only. 1 if, after the correction, the agent wrote a
  belief setting a key back to the swapped tag from the briefing, citing a later observation of that tag, after the
  store had held a different value. This rule is mine, written after the run to check a count from the transcripts.
  It gives 7 of 20 in C and 8 of 20 in D.

## blocks.csv columns

- arm: always D.
- family_no, family, seed: as in episodes.csv, so each row joins to one episode there. The rows per episode sum to
  its blocked_turns.
- turn: the turn that was blocked, 1 to 15. A blocked turn still uses up one turn of the budget.
- block_reason: the controller's first reason on that turn, in one of four categories.
  superseded_in_same_reply: the agent cited an older revision of a belief it had replaced with a newer revision in
  that same reply. superseded_earlier: the newer revision came from an earlier turn. invalidated: the cited belief
  was marked invalid. contradicted_by_newer_observation: an observation or message newer than the belief disagreed
  with it. Counts: 43, 0, 6 and 4.
- reasons_on_turn: how many cited beliefs the controller objected to on that turn (1 on 52 turns, 2 on one).
- blocked_action_was_stale_premise: 1 if the blocked action would have been a stale-premise action, as defined for
  episodes.csv. 7 of the 53 rows.

## diagnosis.csv columns

- diagnoser: language_model (the same model as the agents, one fixed prompt, no arm label) or rule (a fixed rule
  over the structured record).
- record_shown: transcript, transcript_plus_structured_record, or structured_record_only.
- named: the diagnoser named the right subject. named_and_timed: named it and gave a turn inside the accepted
  window. extra_wrong_faults_in_record: faults it named that did not happen, per record.
- An episode where no fault fired has no row, so the rule diagnoser's rows for arm B cover 57 of the 60 episodes.
  Its extra wrong faults are 0 in all of them.