Migration decision.

Collection status and eligibility

All 2,650 scheduled model evaluations across 530 shared cases have first-attempt evidence. 1 required retry has a result, bringing total HTTP attempts to 2651. 2,649 outcomes are resolved; 1 remains unresolved. The original all-five-complete gate did not pass. Recommendations apply only to fully evidenced configurations.

• o4-mini: Complete outcomes, actual charges, timing and verified identity across all 530 cases, with no more primary errors than the incumbent.
• DeepSeek V4 Flash: unresolved outputs; incomplete all attempt pricing; Unresolved or missing full-run evidence; Incomplete full-run charges.
• Qwen3.7-Flash: Complete outcomes, actual charges, timing and verified identity across all 530 cases, with no more primary errors than the incumbent.
• Nemotron 3.5 Lightning: incomplete all attempt pricing; Incomplete full-run charges.
• GLM-5.3-Flash: Complete outcomes, actual charges, timing and verified identity across all 530 cases, with no more primary errors than the incumbent.

Missing evidence never redistributes weights. Tied winners and retaining the incumbent remain options under the declared rule.

Primary quality and agreement

A valid output contains an accepted intent label and may still be wrong. End-to-end agreement uses all planned cases and is the scoring input. Valid-answer agreement excludes failed and unresolved pairs; it must be read alongside output coverage. Arithmetic possible ranges describe unresolved-case limits, not confidence intervals.

Actual cost and cumulative HTTP time

Known full-run reported subtotal: $1.1844958149. Unknown attempted charges: 2. Held uncertainty reserve: $0.006677. The reserve is not a measured charge; the subtotal is not a complete total or final invoice. Per-1,000 cost stays unavailable where attempted charges are unknown.

Cost includes every attempt, including the failed first response. Time is reconstructed per case by adding its recorded HTTP-attempt durations. It excludes automatic cooldown waits and manual pauses between collection segments, so it is not observed end-to-end customer latency. Earlier segments used different client deadlines and timing instrumentation; this is not a controlled model-speed comparison.

Percentiles use linear interpolation over each case’s summed HTTP-attempt durations, including failed first attempts. Automatic cooldowns and manual pauses are excluded. These measurements do not establish production savings, successful-answer latency or a causal model-speed comparison.

Primary failures and paired outcomes

The five paired outcomes partition observed both-valid pairs. Unresolved and invalid answers stay outside this subset. Confirmed errors include wrong labels and unusable outputs; unknown outcomes remain separate. Output-limit truncations are returned unusable responses, distinct from upstream rejections or unknown transport outcomes. A timeout does not reveal the model’s answer or final charge.

The separate 30-case challenge cohort

Challenge cases are not pooled into the primary denominator or weighted recommendation.

Unchanged weighting profiles

Cost and time utilities are incumbent / (incumbent + candidate). Equal values score 0.5; both-zero utility is 0.5. Eligibility requires no more primary errors than the incumbent. Adaptive scores use the final policy-selected result and all-attempt cost and time. Original fixed-cap scores remain separate.

Method, collection segments and configurations

The adaptive policy was declared after operational failures and before inspecting label quality. It preserves every first attempt and permits one prescribed retry after a billed truncation. Pending or budget-refused required retries prevent publication. Original stopped studies remain unchanged.

After a verified upstream rejection, collection paused before continuing with untouched cases. Rejected calls were not retried: they remain failed answers with unknown charges reserved. Automatic cooldowns and a per-model circuit limit changed collection operations, not the model settings, scoring weights or output-limit retry policy.

Public Banking77 cases were reused and publisher labels remain unchanged. These are not customer-adjudicated outcomes, fresh held-out confirmation or a customer rollout decision. The October 10 experiment and historical retrieved-Luna assessment remain separate.

Reasoning can consume the output allowance before a usable label is returned. These are maximum budgets, not measured visible-answer lengths or equal token allowances.

Source identity

Combined summary SHA-256:31d2f1a4ddc87eb9a6682c2c69f51a679d0c51f91bac9279772aa51c3d209b1d

Reporting amendment SHA-256:b940a96188f7adfa9affcc629f668e90be015faea136c126847b76ff1b726da9

Original plan SHA-256:fc86a515c397671c38d6ed40083938075dcc30df692c771de7b660f66a25d193

Untouched-suffix plan SHA-256:fe1274037d6c43988e0f1f8035ec130e731a9df694dd1cbe4a46a4d2d98955a1

Adaptive plan SHA-256:9233745dc9c1a76050b1d2922aac48dcc4454ab2831b20fc29dec5c11b4c91b5

Collection plan SHA-256:fa28afcfde818f2234f7d94fc946db34b2d7254acca6a8fc1b9ea7a829a12c33

Collection terminal SHA-256:3d7f93d43856670f0ca5fb77783384308fab20c25825da2c6e50de11235ce097

Collection amendment SHA-256:6a6da5802d9cdab3a8f08c1e43499fae4436592bba7579394a660daf4cfb46f8

Prepared 2026-10-13 06:43:08 UTC. The decision report, evidence appendix, CSV and curated JSON describe this same audited collection. The manifest pins exact file bytes.