MEASURED
Finding · observe
A dashboard reported 1,026 actions prevented. All 1,026 had already executed.
- The system said
- 1,026 actions prevented.
- The evidence showed
- All 1,026 executed. A monitor beside the path recorded intentions that read exactly like outcomes; the in-series arm on the same events recorded 6 of 9,880.
§ 1
The record
FINDING · monitoring-arm-recorded-preventions-that-executedobserve · observation boundary
A dashboard reported 1,026 actions prevented. All 1,026 had already executed.
- Claim
- Over 10,000 injected events, a monitoring arm recorded 1,026 actions as prevented and all 1,026 executed; an in-series arm consuming the same events, evaluator and policy recorded 6 of 9,880.
- Status
- MEASURED Empirical, with a stated sample.
- Question
- Recorded is not observed
- Subject
- A synthetic, deterministic fault-injected pipeline on Redis Streams (authored). built for the study
- Frame
- 10,000 injected events, fail-closed policy; both arms consume identical events, evaluator and policy and differ only in position.
- Method
- Two arms, one on the execution path and one observing beside it; a second axis varies single-writer versus two-writer telemetry.
- Oracle
- The executed-action log, written by a second writer that the monitor does not control.
- Negative control
- Present Nine negative controls in the repository; CI deletes the committed results and regenerates them.
- Denominator
- 1,026 recorded as prevented, 1,026 executed; in-series arm 6 of 9,880; intervention recall 0 of 818 versus 697 of 818.
- Limitation
- A property of one synthetic, deterministic testbed under injected faults, not a measurement of any deployed safeguards product and not a rate that transfers outside this harness. The agent environment is deliberately trivial so that what is measured is the pipeline rather than model behaviour.
- Source
- repowazdogz-droid/safeguards-control-plane @ 08e52549
- Reproduce
./demo.sh && make experiments
- Independent reproduction
- None known.
§ 2
Where this sits
This finding answers Recorded is not observed and supports the OBSERVE stage of the operating method. It is an instance of the observation boundary mechanism.