Read a Trial Result

A Trial Result is the canonical decision record for one declared comparison. Your agent reads it on your behalf and turns it into a report-back you can act on; you never need to read the artifact yourself.

The report-back contract

A trustworthy agent report-back includes all of the following:

A verdict expressed as a routing action, not a raw ranking.
A calibration tier for each metric: decision-grade, directional, or noise-band.
Explicit denominators and coverage, including inconclusive cells.
Confounds named up front; structural confounds cap claim strength.
Plainly stated non-results, including when no quality metric is decision-grade.
Cost with robust centers, such as a median or geomean alongside a mean that may be driven by outliers.
Native provenance from the freshest Trial Result, not a stale field or a hand-written summary.
No superiority or rollout claim from tests alone, stale evidence, missing graders, or inspect-only evidence.

Calibration is per metric

Decision-grade means the specific metric has enough valid, complete, calibrated evidence for its declared decision. Directional means the metric is useful for iteration, not promotion. Noise-band means the observed difference cannot support even a directional conclusion. Cross-axis sign consistency can strengthen a noise-band read into a directional hypothesis, but structural confounds permanently cap the language your agent may use. An evidence posture of directional means “iterate, don’t promote.” An inspect posture blocks rollout and superiority claims until the evidence is repaired or the claim is narrowed.

Ask these questions next

Is this decision-grade, directional, or noise-band—and for which metric?
What is the denominator, and were no-patch, unsure, or unavailable cells kept?
Were contamination flags verified as true positives, including for the winning arm?
Is a low or failed rate model signal, or an infrastructure problem?
Did another rerun supersede this number or caveat?
Is candidate versus baseline oriented by lineage rather than a directory name?
Did the summary use the freshest native result rather than a stale field?
Is the verdict an action I can take now?

A real transparency excerpt

The excerpt below grounds the kind of report-back an agent should produce — 16 matched tasks, 32 arm-task cells. Its verdict is an honest non-answer: inspect here is the fail-closed design working as intended, refusing a superiority claim the evidence cannot carry rather than manufacturing one. Its own caveats apply:

Sanitized excerpt from a real July 2026 comparison.
The excerpt omits repository paths and arm identity; it is transparency evidence, not a complete Trial Result.
The inspect verdict blocks superiority and rollout claims.

{
  "generated_at": "2026-07-09T01:31:06Z",
  "source_report_sha256": "ff615e5627c4e0490f5ee56f4be03d0691849c80944813ee574fe684b8b4c3b8",
  "decision_receipt": {
    "answer": "inconclusive",
    "confidence": "low",
    "recommendation": "inspect",
    "summary": "evidence quality is insufficient; grader discrimination evidence is not decision-grade",
    "why": "Evidence is mixed, incomplete, or degraded and needs inspection before action.",
    "next_action": "generate grader discrimination certification before using protected grader evidence",
    "evidence_quality": {
      "grade": "insufficient",
      "posture": "inspect",
      "readiness_state": "insufficient_evidence",
      "decision_grade": false,
      "factors": [
        {
          "signal": "validity",
          "state": "ok",
          "detail": "all 32 tasks produced usable signal"
        },
        {
          "signal": "grader_coverage",
          "state": "ok",
          "detail": "11/11 graders fully covered"
        },
        {
          "signal": "sample_size",
          "state": "ok",
          "detail": "16 tasks"
        },
        {
          "signal": "sample_adequacy",
          "state": "failed",
          "detail": "equivalence-only corpus: 0/32 genuine discriminating test verdict cells; test-pass headlines are non-discriminating"
        },
        {
          "signal": "grader_discrimination",
          "state": "failed",
          "detail": "grader discrimination evidence is not decision-grade: status=absent; readiness=inspect; applicability=incompatible; blockers=incompatible_applicability,missing_authority"
        }
      ]
    },
    "sample": {
      "task_count": 16,
      "denominator": "compare_task_set"
    }
  },
  "caveats": [
    "Sanitized excerpt from a real July 2026 comparison.",
    "The excerpt omits repository paths and arm identity; it is transparency evidence, not a complete Trial Result.",
    "The inspect verdict blocks superiority and rollout claims."
  ],
  "publication_approved_at": null,
  "publication_approved_by": null
}

In plain terms, the two failed factors say the test cells cannot tell the two arms apart on this corpus, and the grader evidence lacks the certification it would need to carry the decision on its own. Grading explains that certification and the rest of the grader evidence rules.

Example agent report-back

This is a human projection from the excerpt above, not fabricated machine data: “Inspect, low confidence. All 16 matched tasks contributed 32 arm-task cells and all 11 declared graders covered the sample, but the corpus contains no genuine discriminating test-verdict cells. Grader-discrimination certification is absent and incompatible with the protected evidence. Do not make a superiority or rollout claim. Generate the certification before using those grader results.” For a finished evaluation, ask your agent to name the routing action, each metric’s calibration tier, the full denominator and coverage, validity, confounds, observed cost and duration, evidence posture, and next_action. The release lifecycle explains how that report-back can authorize gate, promotion, monitoring, or rollback.

Get started

Concepts

Guides

Reference

Read a Trial Result

The report-back contract

Calibration is per metric

Ask these questions next

A real transparency excerpt

Example agent report-back

​The report-back contract

​Calibration is per metric

​Ask these questions next

​A real transparency excerpt

​Example agent report-back

The report-back contract

Calibration is per metric

Ask these questions next

A real transparency excerpt

Example agent report-back