Claim Check

One way to engage OMEGA, for when the question is specifically: can this consequential technical claim be independently supported? I build an independent check, show that it catches a planted failure, run it on your pinned version, and report what holds, what fails and what can’t be told yet.

If your question is broader (which approach to pursue, whether an experiment can be de-risked computationally, why a model behaves as it does), bring it as it is. Claim Check is the narrowest form of the work, not all of it.

To start, one sentence: the claim, and the decision that depends on it. Later, the thing itself: a repository and commit, a notebook, a model and its inputs, a results file, or access to a system.

Claims that fit well:

• “Our solver conserves mass on these inputs.”
• “The GPU path gives the same answer as the CPU path.”
• “This proof covers the decoder’s outputs.”
• “This eval score measures refusals correctly.”
• “This result in our paper reproduces from the released code.”

1. Scope. I reply with whether the claim can be tested with what’s available, what access I’d need, how long it will take and a fixed fee. If it can’t be tested, I explain why. That reply costs nothing.
2. Agreement. The wording of the claim, the version, inputs, scope, duration and fee are agreed in writing. An NDA is fine.
3. Early readout. Once access is in place: does the result reproduce at all, and has anything already broken?
4. The check. Built independently of the check that passed. Before I trust a pass, I show the check fails on a known-bad case.
5. Report and walkthrough. A written report in the structure of the sample report, and a call to go through it.

Whether the claim, as agreed, holds on the version and inputs agreed. Usually some mix of:

• recomputing a quantity from raw outputs rather than the tool’s own totals;
• running the same input two ways;
• pushing inputs to the edge of what’s supported;
• planting a fault to see whether your existing check notices;
• checking whether a proof or test actually reads the output you care about.

• The evidence: what ran, on what, with what result.
• Proof the check works: the planted failure it caught.
• The limits: what the result does not show.
• A script to re-run the check on your next release, where the method allows.
• The smallest useful next step, if there is one.

See a sample report, built from a real public case in exactly this structure.

I expect most single claims to take one to two weeks of elapsed time, with the early readout first. That is an estimate based on the public investigations, not a measured average. The duration and a fixed fee are agreed before work starts.

Fails: you hear it privately, before your decision, with the smallest reproducer and a suggested fix. Can’t tell yet: you get the reason (missing access, no independent reference, behaviour outside the agreed scope) and what would settle it. The fee is the same either way.

• Everything you send, and the result, is confidential by default. Nothing is published, and the work is not mentioned, without your written agreement.
• Sensitive data can stay in your environment. I can supply a check for your team to run there and review only its outputs.
• I don’t change a result to suit anyone. You can add context, and it will be included.
• If the check finds a bug in a third-party open-source dependency, you and I agree how and when to report it upstream.

Not certification, accreditation or a regulated audit. Not a penetration test. Not a guarantee about anything beyond the claim, version and inputs checked. I hold no accreditation or security clearance.

Why not just ask our own engineers?

They’re the right people to fix it. An outside check is useful because it doesn’t share the assumptions the original was built on.

Why not ask an AI assistant?

Fine for a first opinion. It doesn’t run your pinned version, plant a fault to test its own check, or stand behind the result.

Each piece of selected research carries one or two labels. They describe the kind of evidence that exists, not how good the work is, and each case states its boundary separately.

Proved

A machine-checked proof exists for the stated model and property.

Measured

A quantity was measured independently, with the instrument itself checked on known answers.

Reproduced

A reported behaviour was reproduced from a minimal, public input.

Counterexample

A concrete input shows the claim is false as stated.

Fixed upstream

The maintainers accepted a fix into the project. Who wrote it is stated on each case.

Reported, no fix claimed

Reported to the maintainers with evidence; no fix is claimed.

Fix submitted, not yet merged

A fix was submitted to the maintainers with its evidence. It is not merged or released.

Model-bounded

True of a model. Whether the model matches an implementation is a separate question.

Scoped

Holds only for the revision, inputs and property named; read the boundary.