Verifier Trust

How Idora makes LLM-backed determinations bounded, replayable, and challengeable · May 2026

Idora uses a language model for semantic verification: evaluating whether code and the execution evidence joined to that code conform to formalized requirements in the context graph. The right question follows: if a language model evaluates code, why should the determination be trusted? The answer is not “trust the model.” The answer is “trust the bounded process.” This page documents that process: what the receipt proves, what it does not prove, and how Idora calibrates the verifier separately from the receipt.

Receipt Integrity

What a receipt proves

A verification receipt is content-addressed. Its identity is the SHA-256 hash of its canonical form. Modify any hashed field and the result is a different receipt with a different ID. The original receipt remains in the graph. Corrections, superseding determinations, and policy exceptions are appended as new receipts rather than edits to the original.

The receipt records what was evaluated, when, against which inputs, by which evaluator, and with what outcome. Specifically: the spec_section and spec_file the requirement comes from, the requirement_text in full, the inputs as path-and-hash pairs (each input may carry a code_provenance field where authorship is known, including agent, human, generated, or mixed provenance), the commit under verification, the evaluator’s model identifier and prompt_version, the determination, the verifier’s confidence, and the structured verification_dimensions reasoning.

What the receipt does not prove: that the evaluator was correct.

The receipt proves that a specific evaluator, running a specific prompt version, against a specific commit and a specific set of input hashes, returned a specific determination. Whether that determination matches the truth of the system is a separate question, answered by the verifier’s reliability profile, not by the receipt’s integrity.

This distinction is deliberate. Conflating receipt integrity with verifier correctness is the failure mode every credible integrity layer must avoid. Receipt integrity and verifier reliability are different layers of the same trust model. Both are necessary.

Receipt integrity and verifier reliability are different layers of the same trust model. Both are necessary.

For example, a release gate response might surface this determination:

Release gate response · example
{
  "determination":    "does_not_conform",
  "claim":            "AUTH-007 at v2.1",
  "proof_chain_id":   "rcpt:7f3e8a2c...",
  "evaluator": {
    "model":          "claude-sonnet-4-20250514",
    "prompt_version": "2026-04-21.1"
  },
  "evidence": {
    "requirement": "AUTH-007",
    "file":        "src/auth/auth.service.ts",
    "commit":      "a3bf...",
    "artifact":    "v2.1"
  }
}

The receipt proves this determination was produced from those inputs by that evaluator at that prompt version. The reliability profile, addressed below, describes how much operational trust to place in that class of determination.

Trust Layers

The four trust layers

Idora’s trust model decomposes into four layers. Each carries a claim, a mechanism, and a current status.

1. Evidence integrity

Claim. Every determination is grounded in evidence whose identity cannot drift.

Mechanism. Receipts are content-addressed. Each receipt’s id is the SHA-256 of its canonical form, and the same identifier surfaces from operational responses as proof_chain_id, making the chain from any decision to its grounding receipts directly traversable. Code inputs are recorded as path-and-hash pairs. Build outputs and artifacts are content-identified by their own hashes. The File node joins verification receipts (which evaluated the file) and execution receipts (which consumed the file) on the file path; the integrity of the join derives from the integrity of the receipts on either side, both already content-addressed.

Status. Operational. Receipts are produced on every push and decomposed into the context graph through the receipt pipeline. Every operational determination surfaced through the product surface traces to one or more such receipts.

2. Determination boundedness

Claim. The verifier returns a closed, structured determination, not free-form prose.

Mechanism. Every verification produces values from a fixed enum, with structured reasoning attached:

Verification output structure
{
  "determination":           "does_not_conform",
  "confidence":              "high",
  "verification_dimensions": {
    "mapping_clarity":      "high",
    "verification_clarity": "high"
  }
}

The public determination vocabulary is closed: conforms, does_not_conform, or indeterminate. Cases outside those states are represented as metadata, evidence, or indeterminate, not as new determination classes. When evidence is insufficient, the verifier returns indeterminate rather than guessing. The shape is fixed: the same structure appears on the receipt and on every tool response that surfaces a determination.

Status. Operational. The determination triple is part of the canonical decision-shape vocabulary on the product surface and is stable across every tool that returns a determination.

3. Replayability

Claim. A determination can be replayed against the same evidence bundle, prompt version, evaluator identifier, and schema.

Mechanism. Every verification receipt records the evaluator (model identifier and prompt_version) and the input set: requirement text, commit, file paths, and SHA-256 hashes at evaluation time. The receipt is itself content-addressed, so a stored receipt cannot drift after the fact. Replaying a verification means reconstructing the same prompt at the recorded prompt_version, addressing the same files at the recorded hashes, and invoking the recorded evaluator or its archived equivalent.

Replayability does not require pretending hosted models are perfectly deterministic, and it does not make the evaluator infallible. It gives Idora and the customer a stable audit procedure: the same evidence bundle, same prompt version, same evaluator identifier, and same schema can be re-run, compared, and challenged. When a prompt version changes, prior determinations remain in the graph addressed by their original prompt_version; new determinations carry the new prompt_version. The historical record preserves comparability across prompt evolution and across model upgrades.

Status. Operational. Every verification receipt produced today carries evaluator and prompt_version fields and pinned input hashes, as specified in the receipt schema documented in the architecture overview.

4. Reliability calibration

Claim. The verifier’s reliability is measured separately from the receipt and is itself a reportable artifact.

Mechanism. The verifier records two structured reasoning dimensions on every verification, mapping_clarity and verification_clarity, anchored to a calibrated band system with anti-inflation discipline. Both dimensions are surfaced on the receipt and on every tool response that returns a determination, so callers see not only the outcome but the structured reasoning behind it.

Reliability beyond per-determination dimensions is a separate measurement. Today, production receipts carry per-determination clarity dimensions on every verification. The Verifier Reliability Card, when published, will aggregate verifier performance across a stable conformance benchmark and make that aggregate reliability reportable by evaluator and prompt version, with documented disagreement protocols for cases where Idora’s determinations diverge from a partner’s expectations. Idora commits to publishing the first Reliability Card alongside the conformance benchmark when both reach a state worth reporting.

Status. Mapping clarity and verification clarity are operational, recorded on every verification. The Verifier Reliability Card and the conformance benchmark are commitments, not yet operational.

Clarity Dimensions

Two clarity dimensions

The verifier records two dimensions of structured reasoning on every determination, distinguished because they fail in different ways and for different reasons.

mapping_clarity asks whether the code maps to the requirement at all. It captures whether the verifier could find the implementation surface the requirement governs. A high mapping_clarity means the verifier identified the relevant code with confidence. A low mapping_clarity means the verifier was uncertain whether the code it inspected was the right code, regardless of what that code did.

verification_clarity asks whether, given the mapped code, the code satisfies the requirement. It captures whether the verifier could reach a determination from the inspected code. A high verification_clarity means the code’s behavior with respect to the requirement was unambiguous on inspection. A low verification_clarity means the requirement and the code interact in ways the verifier could not resolve from the available evidence.

Recording the dimensions separately surfaces a distinction that a single confidence score collapses. A conforms determination with high mapping_clarity but low verification_clarity reads differently from one with low mapping_clarity but high verification_clarity. The first means the verifier found the right code but was unsure whether it satisfied the requirement; the second means the verifier was unsure whether it found the right code at all. Different remediation, different epistemic state, different reliability profile. The receipt records both so the reader can separate them.

Failure Modes

Common failure modes

Verifier failures fall into known classes. Idora’s trust model is designed to make these failures visible, not impossible.

  • False conform. The verifier returns conforms when the requirement is not actually satisfied.
  • False blocker. The verifier returns does_not_conform when the requirement is satisfied.
  • Incorrect mapping. The verifier evaluates the wrong file, seam, or implementation path.
  • Over-trust in tests. Passing tests are treated as stronger evidence than they deserve. Tests demonstrate behavior on specific inputs, not satisfaction of a requirement.
  • Missing runtime evidence. Code appears conforming, but behavior depends on configuration, feature flags, environment state, or services outside the current graph boundary.
  • Ambiguous requirement. The requirement does not contain enough information for a reliable determination.
  • Adversarial compliance. The implementation appears to satisfy the wording while violating the intent.

The indeterminate state, the clarity dimensions, replayable receipts, the challenge protocol, and future Reliability Cards all exist to keep these failure modes from hiding behind a confident answer. Visibility is the trust mechanism.

Bounded Claims

What Idora does not claim

The trust model is bounded by what receipts can carry and what the verifier can evaluate. Stating those bounds is part of the model.

  • Idora does not claim that passing tests prove conformance. Tests are execution evidence; a passing build that consumed a file does not establish that the file satisfies any requirement.
  • Idora does not claim that a determination from the verifier proves software correctness. A conforms determination is bounded by the inputs the verifier inspected and the requirement it evaluated. Behavior outside that scope is outside the determination.
  • Idora does not claim to verify decisions that were never formalized. Decisions that exist only in chat threads or meetings are out of scope; the verifier evaluates against formalized requirements, and the receipts trace to those requirements specifically.
  • Idora does not claim to capture runtime configuration, feature flags, cloud-console mutations, or cross-repo behavior unless those evidence sources have been ingested into the context graph. Mechanisms that live outside files are outside Layer 1 scope.
  • Idora does not claim that receipts make verifier reliability automatic. Receipt integrity proves the chain of evidence; verifier reliability is measured separately and reported separately.

What Idora does claim, in one sentence: given formalized requirements and the evidence currently in the context graph, Idora returns a bounded, replayable determination with a receipt chain showing exactly what supports it and what remains outside scope.

Challenge Protocol

Design partner challenge protocol

Idora’s verifier is in active calibration. Design partners are invited to challenge it directly.

Partners can bring known production requirements and known incidents to Idora. Idora runs them through the verification surface, returns the determinations the verifier produces against the partner’s evidence, and walks through the resulting receipt chains with the partner. Where the partner’s expected outcome and Idora’s determination diverge, the disagreement becomes an input to the conformance benchmark Idora is building. The receipts, the prompt versions, and the structured reasoning are all on the table.

This protocol is operational today as part of design partner engagement. To bring requirements and incidents to the verifier, contact engineering@idora.dev.

The framing is specific. Idora is not asking partners to trust the verifier in advance. Idora is inviting partners to challenge it on cases where the partner already knows the answer. The reliability profile is built from the disagreements the verifier survives and the disagreements the verifier loses, not from claims about the verifier in the abstract.

Idora is not asking partners to trust the verifier in advance. Idora is inviting partners to challenge it on cases where the partner already knows the answer.

Trust Boundary

The trust boundary

Compressed:

The receipt proves the evidence chain.
The reliability profile describes how the evaluator has performed for this class of determination.
The metadata describes graph maturity at the time of the answer.
The indeterminate state prevents Idora from inventing certainty when evidence is insufficient.

That is the trust model.

contact · engineering@idora.dev