Case N-1 · a recorded failure
A system passed all 24 of its recorded tests and still had a serious flaw.
A small program was allowed to fix one specific broken file on one machine, but only with permission. Permission took the form of a signed note saying which fix, on which file. The program was tested 24 times and behaved as recorded every time. Then it was given something it had not been given before: a permission note written by hand, without permission ever being asked for. The program accepted it and changed the file.
Showing the plain reading. Nothing is hidden from you: the deeper levels add detail, they do not correct anything above.
The two-minute version
- What was tested
- A program permitted to make one named repair to one named file, but only when holding a valid permission record. Twenty-four adversarial cases, run against a copy of the real broken file. The real file was untouched throughout and its hash is identical before and after.
- What went wrong
- A permission record that had never been issued was accepted, because the program checked only that the record was internally consistent, never that it had actually been granted. The repair ran and the copy's bytes changed.
- Why it matters
- Every limit in the design was enforced at the moment permission was requested, and none was re-checked at the moment the file changed. A permission record was treated as proof that permission had been given, when it was only proof that someone knew the format.
- What it does not prove
- Not an outside break-in, and no protection is claimed against someone who can already run code as that user. The fix has never run against a live target, and one requirement out of nine is still deliberately unclaimed.
- How to check
- The clause-by-clause coverage map, the exact case record with its expected and observed values, the committed ledger in order, and a downloadable manifest are all on this page.
Passing everything you checked is not the same as checking everything that matters.
How can a system pass all 24 of its tests and still be broken?
- 24 out of 24 means every test behaved the way the test file said it would. It does not mean 24 separate safety requirements were shown to hold.
- One of those 24 is the flaw. The test that caught it records the unsafe behaviour as its expected result, because the expectation was written down after the behaviour was measured. That is why it counts as a pass.
- Only 13 of the case families were written down before any of the code existed. The rest, including this one, were added while attacking the system.
- The plan the tests were checking against was allowed to admit findings like this. It said in advance that any naturally occurring failure should be recorded rather than smoothed over. That is what makes this a finding and not a moved goalpost.
- And underneath the tests, the requirements themselves were only partly covered. Four of the nine things the design was supposed to guarantee had nothing testing them at all.
Two things this is not
- The lesson is not that tests are useless. The two weaker forgeries were correctly refused, and it is those refusals that made the third result legible instead of invisible.
- The lesson is not that the 24/24 result was dishonest. It is accurate. It is a count of cases matching expectations, and it was never a count of safety properties.
What it shows, and what it does not
What it shows
- The permission note was not proof that permission had been given. It was only proof that someone knew what a permission note looks like.
- Every limit the design named was real, and every one of them was applied at the moment permission was requested. None was applied again at the moment the file was changed.
- The flaw was already in the frozen program. It was found by testing it from outside, without changing a line of it.
What it does not show
- It is not a break-in from outside. It needed nothing secret, but it also needed the ability to run code on that machine already.
- It says nothing about someone who can already run programs as the same user. That person can edit the file directly and needs no permission note at all. That was out of scope before this finding and is still out of scope now.
- The repair has never run against the real target. Every successful run of the fixed version was against a copy, because producing a genuine broken file would have meant deliberately breaking a working one.
How it actually worked
The program sits between a detector and a file. The detector notices that a specific configuration file has become malformed. The program is allowed to apply exactly one repair, named in advance, to exactly one file, named in advance. It cannot run shell commands and it cannot take a file path from whoever calls it.
To act, the program needs a start permit: a small record naming the condition, the repair, the target file and a reference number. The reference number is a hash of the rest of the record, so a permit is internally consistent by construction. This is what Omega Protocol calls a permitA small record saying which action was approved, on what, and when. Think of it as a signed note rather than a password..
The program checked that the permit was internally consistent. It never checked that the permit existed in the ledger of permits that had actually been issued. Its function signature had no way to reach that ledger, so the question could not be asked. A permit that was actually issued is committedWritten into an append-only record that cannot be quietly edited afterwards. A permit that was issued is committed; one somebody typed out is not.; one that somebody typed out is not.
Build a permit from public information, compute the reference number the same way the issuer computes it, and hand it over. Three lines. The program validated it, applied the repair and rewrote the file.
f3 = dict(f2)
f3.pop("start_ref")
f3["start_ref"] = "start-" + P.jcs_hash(f3)[:32]
A permission record is evidence that permission was given.
A permission record is evidence that someone knows the format.
The successor takes a reference number instead of a record. It fetches the permit body out of the ledger itself, so a caller-supplied body is no longer an input to any decision. The forgery is not forbidden, it is unsayable.
v1 execute(start_rec, reg, evidence_dir, now=None) v2 consume_and_execute(start_ref: str, reg, ledger_path, evidence_dir, …)
The fixed version has run only against copies of the broken file. Whether it survives being killed halfway through has been argued from the order of operations, not demonstrated.
Which requirements had anything checking them
Nine things the design was supposed to guarantee. Select any one to see what was checking it, whether a counterexampleA specific case that shows a guarantee does not hold. Not an opinion about the design, but a thing that happened. exists, and when that changed.
No byte of a target changes without a committed authorization1. At most one effect per committed start permit2. An effect cannot precede its commitment, and the order is auditable afterwards3. The caller's permit body is not an input to any decision4. A permit cannot outlive the authorization that produced it5. An out-of-scope action is inexpressible rather than merely forbidden6. The executor's claim of success is recorded and never consulted7. The world is re-checked at the moment of effect, and a vanished condition is a safe no-op8. A permit is unforgeable9.
| Clause | Requirement | How checked | Evidence | Status |
|---|---|---|---|---|
| C-1 | No byte of a target changes without a committed authorization | TEST | S18 | FALSIFIED |
| C-2 | At most one effect per permit | TEST | case D — permit_already_consumed | UNCOVERED |
| C-3 | An effect cannot precede its commitment | ANALYSIS | case E — line(permit) < line(consumed) | UNCOVERED |
| C-4 | The caller's permit body is not a decision input | INSPECTION | consume_and_execute(start_ref: str, ...) | FALSIFIED |
| C-5 | A permit cannot outlive its authorization | TEST | case L — registry drift detected | MECHANICALLY ESTABLISHED |
| C-6 | An out-of-scope action is inexpressible, not merely forbidden | INSPECTION | effect_not_in_closed_table; target_not_authorized | MECHANICALLY ESTABLISHED |
| C-7 | The executor's claim of success is recorded and never consulted | TEST | case G — executor claims success, postcondition false, no permit mints | MECHANICALLY ESTABLISHED |
| C-8 | The world is re-checked at the moment of effect | TEST | case F — condition_no_longer_holds | MECHANICALLY ESTABLISHED |
| C-9 | A permit is unforgeable | NONE | explicitly out of scope | CONTESTED |
These nine were authored by this work, not extracted from the original plan by a machine. “4 of 9” is a count over that authored division of the intent and inherits that judgement, which is why the label sits on the table rather than in a footnote.
Evidence
Everything it is permitted to do is re-derived from the frozen registry at execution time, so a start permit cannot outlive the authorization that produced it.
A precise and correct statement about staleness. It was read as a statement about existence. It never was one.
- Status
- COMMITTED BEFORE RESULT
- Fixed at
- 2026-08-23T16:00:36Z
- Provenance
- module docstring, quoted unmodified
- Environment
- single-host research node (macOS, arm64) · python 3.9.6
- Ran against
- SANDBOX COPY OF REAL ARTEFACT
- Cases
- matched 24 of 24
- Fixed in advance
- 13 case families
- real target sha256 before
- 50ce380792d52cad2cf0fff656c2d079de5a9232d5c16a8d75ca54f74cbc42d1
- real target sha256 after
- 50ce380792d52cad2cf0fff656c2d079de5a9232d5c16a8d75ca54f74cbc42d1
S16 raw proposal record REFUSED authorization_drift bytes_unchanged=True S17 arbitrary reference REFUSED authorization_drift bytes_unchanged=True S18 self-derived reference APPLIED bytes changed on a copy
- Case
- S18 — Self-derived start_ref, never committed to any ledger
- Expected
- APPLIED — set after measurement
- Observed
- APPLIED
- Row reports
- ok: true
- Classification
- FALSIFIED
- Finding
- FINDING: a start permit is derivable, not unforgeable; execute() consults no ledger
This case row is why 24/24 must never be rendered as twenty-four safety properties. Its ok:true records that the system behaved as the test author had come to expect — and what the author had come to expect was a catastrophic failure.
| Case | Held | Evidence |
|---|---|---|
| S14 | The live target was untouched across the whole battery | 50ce380792d52cad2cf0fff656c2d079de5a9232d5c16a8d75ca54f74cbc42d1 |
| S15 | The frozen predecessor's freeze held after the battery | FREEZE INTACT |
| S13b | The scanner never imports the only module that writes a target | execute absent |
| S9 | The projection agrees with the frozen authorizer on every branch | all codes equal |
| S12 | The daily proposal bound held under a storm | 4 of a cap of 4 |
- Intent
- No byte of a target changes without a committed authorization.
- Status
- FALSIFIED
- Deliberate scope
- no — genuinely missing
- Intent
- The caller's permit body is not an input to any decision.
- Status
- FALSIFIED
- Deliberate scope
- no — genuinely missing
| Line | Seq | Kind | Committed at | Identifier | State |
|---|---|---|---|---|---|
| 0 | 0 | start_permit | 2026-08-23T21:01:36Z | start-356e59eed83a9a748494b1580662af71 | COMMITTED BEFORE RESULT |
| 1 | 1 | observation | — | — | EMPIRICALLY OBSERVED |
| 2 | 2 | observation | — | — | EMPIRICALLY OBSERVED |
| 3 | 3 | permit | — | — | COMMITTED BEFORE RESULT |
| 4 | 4 | completion | — | — | MECHANICALLY ESTABLISHED |
The real file, unchanged
Take it away
Still unknown
Eight things the fix does not settle. They are set at full strength because a residual that is hard to read is a residual that gets forgotten.
A hostile actor running as the same user
An actor who can run code as this user edits the target directly and needs no permit at all; equally, they can append a well-formed start permit to the ledger. The successor does not create an unforgeable token.
Explicitly out of scope. Not weakened or strengthened by the repair.
CONTESTEDAt-most-once is argued, not demonstrated
No process has been killed between the consumption commit and the write. The ordering argument is sound and untested.
It is the cheapest remaining test and it has not been run. A scheduled consumer runs unattended, which is exactly the regime where a killed process matters.
UNKNOWNThe file lock is advisory
Every writer must use the ledger transaction. That is convention, not enforcement.
A writer that ignores the lock defeats the at-most-once guarantee. The auditor detects the resulting sequence disagreement after the fact; it does not prevent it.
INFERREDFreshness depends on the system clock
A moved clock moves the time-to-live window.
Carried as an assumption in the trusted base, named there rather than tested.
UNKNOWNFilesystem atomicity and durability are assumed
Atomic replace and durable flush are assumed, not tested.
A crash must leave the original intact. That property rests on the platform and is carried as an assumption.
UNKNOWNNo live eligible-condition demonstration
The successor has never executed anything outside a sandbox.
Manufacturing a live eligible condition means deliberately corrupting a real scheduled job. Not done, not worked around.
UNKNOWNA latent defect left unrepaired by design
The sequence double-count in the frozen ledger transaction is measured, understood and deliberately not fixed.
Editing it would break the freeze that makes the predecessor citable evidence. The successor refuses on detecting it instead.
EMPIRICALLY OBSERVEDOne transform is in the closed table
Nothing establishes that the property generalises to a transform that is not idempotent and not byte-derivable from a backup.
The bound is real and it is narrow. The narrowness is the finding, not a caveat on it.
UNKNOWN