Calibration is now public

When an agent signs an onchain trade for you, the question that matters is rarely answered: on what evidence did it decide the route was any good, and was that evidence ever checked against what happened? Today Routescore makes its answer public. The /calibration surface is live.

Routescore is a read-only, pre-sign evidence element for agentic DeFi. It produces a modeled route score with caveats and records the decision so it can be reviewed later. Calibration is the accountability record underneath that.

What just went live

The calibration surface publishes how well three source-specific baseline forecasts — settlement_cost_baseline.v1, onchain_quote_dispersion.v1, and routing_leak_baseline.v1 — track their own measured quantities. To be clear about scope: this checks those baseline forecasts, not the composite 0–100 Routescore. It reports two standard, probability-calibration numbers: a Brier score (are the probabilities right?) and expected calibration error (when we say 30%, is it really about 30%?).

The three sources measure different things in different ways: settled execution cost from real executed swaps, reconstructed historical on-chain quote dispersion (a real-but-hypothetical QuoterV2 reconstruction, not an execution), and subsequently-observed cross-aggregator dispersion across 1inch, 0x, and CoW. Three acquisition paths reduce single-source dependence — though two share the same Ethereum state and pools, so this does not eliminate common-mode failure, manipulation, or model error.

A slice of routes — a cohort, reported one dimension at a time (by chain, venue, token pair, size bucket, freshness, or source) — earns a public claim only after a strict gate: at least two independent sources, each with ≥ 30 confirmed labels and ≥ 30 probability-eligible pairs, at label confidence ≥ 0.6, reaching Brier ≤ 0.2 and ECE ≤ 0.1 on its own outcomes, where a "bad outcome" is the source's own measured quantity reaching ≥ 30 bps (an incurred loss for settled execution cost; a ≥ 30 bps dispersion, not an incurred loss, for the two dispersion sources) and the source measures a known quantity. At launch, a first set of cohorts has cleared this bar.

What you can do with it today

• See the track record before you weigh the score. Open a cohort that has cleared the gate and read how the baseline forecasts matched measured outcomes.
• Check the provenance. Every public-eligible claim is bound to a recorded source-set version and content hash, so a published number can be cited and challenged against that fixed, recorded reference. (The version and hash are exposed today; a downloadable observation bundle is not, so this is provenance to cite and dispute, not a one-command offline re-derivation.)
• Read a route. See a modeled score with its caveats attached, so an agent — or you — has evidence, not just an assertion, before signing. The raw snapshot is at /api/calibration.

One honest caveat

Calibration tests probability calibration only — not whether a score ranks routes well or would have saved money on any trade. Modeled is not guaranteed. A cohort that has not cleared the gate still shows its diagnostic Brier, ECE, and counts, carries a "No public claim" badge, and is not designated public-eligible — the numbers stay visible, and illustrative figures are never presented as live. A drift monitor runs forward: today, if it flags a degradation, all public claims in the snapshot are withdrawn together until they recover (per-cohort drift is a future enhancement). Expect covered cohorts to grow as more measured outcomes accrue.

Come check our work

If verifiable evidence is useful to how you or your agents make onchain decisions, take a look. Open /calibration, pick a cleared cohort, and check the source-set version and hash it is bound to; the methodology and limitations are public too. Routescore is one composable, read-only evidence element in the stack — it models exposure and records the decision; it does not execute, route, sign, or guarantee anything. The part we ask you to weigh is the part you can inspect for yourself.