How the check works, including where it fails.
The mechanism underneath every system we build, written for someone who will evaluate it rather than be reassured by it.
Every claim the system emits carries the source it came from, or it is not emitted.
That is a design rule, not a guarantee about the world. It says what the system is permitted to do, which is falsifiable — you can look at any output and check whether the rule held.
Nothing leaves without its source
A guarantee that nothing is ever wrong is not a claim an engineer believes. A rule about what is allowed to leave the system is.
- Extraction is separated from assertionThe step that reads a document and the step that states a fact are different steps with a checkable boundary between them. Most failures we have seen come from collapsing the two.
- The check runs against the retrieved textNot against a summary, an embedding or the model's memory of the page. The stored source line is the artefact the check uses and the artefact you get.
- Failures are recorded, not swallowedA dropped claim produces a record with the reason. The rejection log is as much a deliverable as the output.
- A person is in the path where it mattersEvery system has a defined point where a human approves or overrides, and a kill switch that does not require us.
- The claim gate is deterministic, pre-send and fail-closedNo model judgement anywhere in that path. Every factual claim in an outgoing message must resolve to a stored source; anything that does not resolve is held with the reason recorded. It is channel-aware, because a postal address belongs in an email footer and is nonsense in a LinkedIn message. One definition of the rule, imported by every gate — two copies of a safety rule is how one gets fixed and the other does not.
- Scoring is separated from the fit decisionFit is a gate and it is binary. Score is an ordering over what passed, and it is a different model with a different failure mode. Collapsing the two is how a ranking quietly becomes a qualification, and then nobody can say why an account is in the list.
The failure modes we know about.
Published because a system whose failure cases you cannot see is one you have to take on trust — and because these are the cases that decide whether a build is worth doing at all.
- Ambiguous self-descriptionA firm that describes itself in language that is true of two different business models. The classifier's job here is to abstain, not to guess — and abstention has to be a first-class verdict or the error rate is a fiction.
- The page that no longer existsA cited source that changes or 404s after capture. The stored snapshot is what the evidence file contains, with the retrieval date, precisely so this is visible rather than silent.
- Correct extraction, wrong ruleThe system read the page correctly and the encoded rule was wrong. This is the most common real failure and the reason the rule set is written with you and versioned.
The model proposes. The rules dispose.
The boundary is published because it is the thing a competitor cannot copy by rewording a page — they would have to rebuild.
A model that decides cannot be audited, because the reason changes between runs.
- What the model doesReads source documents and pages, extracts candidate claims, drafts language from facts already verified, and classifies against rules written with you.
- What the model never doesDecide. Every verdict, every threshold and every gate is deterministic code. The model proposes; the rules dispose.
- Why it is built that wayA model that decides cannot be audited, because the reason changes between runs. A model that extracts and a rule that decides gives you an answer you can argue with, which is the only kind worth having.
What we have not published yet
A measured accuracy figure. What exists today is an agreement rate between two models across a 36-firm sample, and inter-model agreement is not accuracy — publishing it as though it were would be exactly the kind of number this page exists to argue against. A hand-labelled gold set of 100–150 firms is the work that makes a real figure publishable, and it is on the list.
Four of these systems run on our own account. See them on the Financial & investment firms page
- Client-hosted by defaultSystems run in your environment, on your infrastructure, under your accounts. There is no OPSPHI platform in the middle.
- What leaves your networkNothing by design. Where a third-party model or API is used, it is your account and your contract, and the boundary is written down before we build.
- Logs and replayEvery input, decision and output is logged and replayable, so any past number can be reconstructed exactly.
- AccessNamed individuals only, disclosed before the engagement, removed at handover.
- RetentionWe hold nothing of yours after transfer other than what you ask us to hold for maintenance.
- Sub-processorsListed with what each one sees.