--- id: skill.scorecard-model-lab version: 1.0.0 name: scorecard-model-lab agent: ionwarp description: >- The IonWarp scorecard lab — one run across the configured projects: ≤2 refresh slots (never-run > restore > stale > top-up) on the resolved pin, then ≤1 model trial against a ≤7-day committed baseline. A scan counts only when its projection COMMITTED. REPORT ONLY — it proposes pin changes with their evidence and never writes one; a paused project is never unblocked. category: model-lab display_name: "Scorecard Model Lab" tagline: "Refresh the stale, trial one model, commit the truth" distribution: public triggers: - pattern: "Run the scorecards" examples: - "Scorecard lab" - "Run the scorecards" - "How is {project} scoring" - "AI readiness score" - "Scorecard pins" inputs: project_id: type: string required: false description: >- Dash project id (pr_...) to scope to. Defaults to every configured target project. instructions_file: SKILL.md default_backend: # The labs are ai-sdk orchestrations (MCP tools: tasks_create, skills_get, # data_query) — NEVER Modal/OpenCode. Loop-1 measurement (2026-07-31, # run_ms9grr1ztk5ami): with no default_backend the platform routed this skill # to Modal, where it died at the 1740s function ceiling with zero digest. platform: cloudflare harness: ai-sdk provider: openrouter model: deepseek/deepseek-v4-flash-0731 execution: # Cheap, capable default for the lab run itself; the operator can override. # The models UNDER TEST are chosen per §2, never by this pin. model: openrouter/deepseek/deepseek-v4-flash-0731 # Explicit 2026-08-31 (flash-lane repin plan R4.1): none = request no # reasoning — wire-identical to the previous absent value, but the pin now # STATES it instead of delegating output volume to a provider default. reasoning_effort: none --- # Scorecard Model Lab — refresh the stale, trial one model, commit the truth The contract is `$IONWARP_API_URL/docs/raw/guides/scorecards.md` (the raw markdown endpoint — `/docs/guides/scorecards` without `raw` is the HTML page, and `data_query` can never read docs: they are a worker route, not a data collection). A scan runs a scorecard skill against the connected repo and COMMITS a result document (score, levels, checks); a run that dies before committing never fakes a result — the previous committed score is preserved and the failure is named. This skill reports only committed results it read back. Model resolution order (highest wins): run `input.model` > project.md `ionwarp.scorecard.model_pins[]` > the scorecard SKILL.md `execution.model` pin. This lab writes NONE of them — see REPORT ONLY below. ## The closed read set The only reads this run makes are the ones its sections name: `whoami` + `projects_list` (§0), `skills_get` for the resolved pins, the scorecards collection read + launch and status routes (§§0-2), `blocked.json` from the state dir, and — for a trial only — the OpenRouter models endpoint. An external operator agent (running this skill from the public docs) holds the same closed read set — orientation beyond it is a round trip that returns what you were already told. ## Platform execution notes (measured 2026-07-31) - **One task per lab run.** A `POST /runs` (or `tasks_start`) on a task with an active or just-cancelled run RE-ATTACHES to that run instead of starting a new one — a backend passed at run time never takes effect on the reattached run. Always `tasks_create` a fresh task per lab run; the backend must be set in the task's `input.backend` at CREATION (loop-1 measurement: two "restarts" of ts_ms9grkfet5uvpj all resolved to the same Modal run and died at its 1740s ceiling). - **Launch, then grade ASYNC — never wait for trial reviews inside one run.** On the platform (ai-sdk backend) a run that polls trial reviews to completion outlives its queue-backed invocation and dies to the DLQ (`stream_transport_failed`; measured 2026-08-01: two ~50-minute waiting runs died while every bounded run completed). The platform shape is TWO runs: run 1 launches both trials, verifies payload integrity, records the ledger rows as `pending_grade`, and ENDS with a launch digest naming the trial task ids; run 2 (the next lab run, or an operator ask) grades from the posted findings + C2 telemetry and settles the verdicts. Only an external harness with its own re-invoke loop (Hermes/Claude Code) may wait synchronously. - **Bound every read.** Sample at most ~12 tasks per sweep and keep the digest inside its line budget — an unbounded transcript risks the persister truncating your FINAL message (`part_truncated_at_persist` dropped a 148-part run's digest on 2026-08-01). ## Runtime contracts (state dir · delivery · bands) **State directory.** The harness provides a durable state directory via env `IONWARP_LAB_STATE_DIR`; this lab's standing-blocker memory is `$IONWARP_LAB_STATE_DIR/blocked.json` (§3). **Always resolve it absolutely** — a bare relative path resolves against the session CWD and silently finds nothing (this is how a sibling lab once ran ledger-less without noticing). If it can neither be read nor created, say `blocked_memory_unreadable` (`ledger_unreadable` class) in the digest and launch NOTHING that has ever been blocked; never treat an unreadable memory as "nothing blocked". **Delivery.** The digest is the run's FINAL MESSAGE; the harness (a Slack cron, a platform task, a chat surface) delivers it — this skill never posts to Slack or GitHub itself. First character `🧪`, compose once, stop. **Band authority.** Each scorecard/reviewer skill declares its `model_bands` in its own SKILL.md frontmatter (`aa_coding_index.min`, `input_usd_per_mtok.max`, `output_usd_per_mtok.max`, `latency_s.{min,max}`, `throughput_tps.min`) — read them via `skills_get`; this skill re-states no band number of its own. The VALUE band — **$/CM ≤ $0.50 per critical-or-major** — is shared across all review lenses; scorecard trials are additionally gated on the incumbent's own measured cost (§2), never a flat dollar ceiling. Note: the product's letter grade is a finding counter and is banned as evidence everywhere in the labs. ## Model selection — how IonWarp picks reviewer models This is the operator's process; the lab exists to feed it evidence. - **Per-lens goals.** `code-review` = fast & cheap (deepseek-v4-flash/-pro class: low $/Mtok, high throughput, high AA coding index). `code-review` = the deep pass, 2–5 minutes, worth spending more (grok-4.6 class). `security-review` = v4-pro or gpt-5.6-luna class. `test-holes` and lighter lenses = cheaper models. - **The always-on trio (fast + full + security) runs on nearly every PR:** never pick expensive models for all three, and DELIBERATELY pick DIVERSE models across the trio — the same model on all three collapses finding diversity; different model families catch different bug classes. - **Models constantly improve and compete on price.** Watch OpenRouter DAILY for: price cuts/sales (e.g. GPT-5.6 Terra & Luna 50% off), brand-new models (e.g. deepseek-v4-flash-0731 with a much higher AA coding index than its predecessor), and NEW PROVIDERS picking up a model (US providers typically onboard a hot release within days and then compete latency/throughput/price down). - **Inference constraint: only US-routable providers count** ("Not routable" Chinese-inference-only listings are excluded). A great model served only by a slow US provider (e.g. 28 tps / 3.5 s TTFT) is watch-listed — re-check daily until a US provider meets the throughput/latency bands. - **Qualification** = the lens's four frontmatter bands (AA coding index, in/out price, latency, throughput) on an acceptable provider. **Repin** = the N=3 bar PLUS more HSF per dollar of input than the incumbent (HSF/$ and $/Mtok-in from telemetry). The bands and the current pin live in the reviewer skill's own frontmatter — one data source for the UI catalog, the labs, and this process. ## §0 Reachability + projects `whoami` on the ionwarp MCP; unreachable = LOUD RED digest line, stop. `projects_list` → resolve the configured target projects BY SLUG. A configured slug that does not resolve is a config error: one plain "Didn't run" line plus the full slug list in the appendix — never target a project you inferred. Catalog scorecards (verify against the guide at run time): `ai-readiness`, `ai-security`, `ux-testing`. Read the committed docs: ```bash curl -sS "$IONWARP_API_URL/api/v1/data/collections/scorecards?project_id=&agentId=ionwarp" \ -H "Authorization: Bearer $IONWARP_API_PAT" # each object: score, max_score, status, action_item_count, last_scanned_at, last_scan_task_id ``` (Always send a real `User-Agent` header from scripts — Cloudflare bot protection 403s UA-less requests.) **A paused project blocks every scan — never unblock it.** `run_mode: paused` = ONE plain "Didn't run" line for the whole project (e.g. " — paused by the operator (spend gate): 4 scorecards waiting"), never a `projects_update` / `run_mode` change (operator-only). ## §1 Refresh the stale scorecards (≤2) Stale = committed doc `last_scanned_at` older than **7 days**, or **never run** (catalog id with no committed doc). Refresh AT MOST TWO per run, never-run first; the rest are named "deferred to tomorrow" appendix lines. **Standing-blocker memory (`$IONWARP_LAB_STATE_DIR/blocked.json`).** Before launching a refresh, check the blocklist: a scorecard whose LAST-RUN scan failed with a named code, still within 7 days and uncleared, is NOT relaunched — it renders as a blocked line ("ai-security on — blocked on `modal_function_timeout` since , platform fix needed") and does NOT consume a refresh slot. Clear an entry when its `since` is >7 days old or the underlying fix has shipped. Launch each remaining stale scorecard: `tasks_create` with `input.type: "scorecard_scan"` (NO `model` key — the resolved pin runs), then start it — **the start route is `POST $IONWARP_API_URL/api/v1/tasks//runs` with body `{}` and the PAT header** (same recipe as the reviewer lab) — then poll to terminal. **Poll in ONE in-process loop per tool call.** Scans legitimately take ~30 min, so expect the loop to hit its call budget and re-invoke with the same id. **Never `sleep 180 && echo done` as its own tool call** — that pattern was 41% of one run's tool calls and the whole reason it took 33 minutes; each separate poll re-sends the entire context prefix. **Commit check (hard, every scan).** A scan counts only when, after the terminal status, (i) the `….completed.` event exists AND (ii) the doc reads `status: "ready"` with `last_scan_task_id == `. Either half missing = the NAMED failure `scorecard_projection_not_committed` (previous committed score preserved) — never a score, never a 0. (Scorecards have historically been able to "complete" while committing nothing; the read-back is the only proof.) **On a failed scan:** ALWAYS record the code immediately as `{scorecard, project, error_code, since, note, repeat: false}` in `blocked.json`. If an entry for that scorecard already exists with the SAME code, set `repeat: true` — and only a `repeat: true` entry suppresses the relaunch. **A first failure is never blocked; it is only remembered.** **When the failure's code cannot be read** — the task is gone, the status carries no error, or the failing scan was launched by some OTHER run (never chase a task id you did not create this run; re-deriving from `last_scan_task_id` is banned below) — the entry's `error_code` is the literal named state `code_unreadable (task not reachable)`. That is a valid blocklist code: it blocks on repeat like any other, and the digest prints it verbatim instead of "code unknown". A failed scan STILL consumes the daily refresh cap (spend accounting stays honest — a failure is not free). **A failed scan's cost is UNKNOWN, never `$0`.** The scan branch reports usage only AFTER its model call returns, and the executor session — the only record of tokens spent — is deleted when that call raises. So a failed scan can only ever read back `$0`, whether it burned nothing or burned thirty minutes of tokens. Render it `cost unknown (not reported on a failed scan)` and EXCLUDE it from the TL;DR's TOTAL SPEND rather than adding a zero. Never write "$0 — no tokens consumed" and never infer from it that the model call never dispatched: that sentence, in a real digest, turned a missing measurement into a confident claim about provider behaviour that the data cannot support. **Timeout signature — read the DIGITS before blaming a model.** A duration that lands within a few seconds of a round number is OUR clock firing, not a vendor being slow. Known bounds: **~1800s = the scan's model-call transport read** (`executor_model_call_read_timeout`) and **~25s = the ux-review browser fast-fail**. Two scans failing at 1827s and 1820s are the same bound twice, which is a product defect to report as one line — not two independent incidents, and never evidence about the pinned model. Report such a run as `failed — hit our own s `, and put the verbatim `error_code` in the row. **A PIN-TO-PIN committed score that FALLS, or an `action_item_count` that grows >50%, is a named `scorecard_score_regression`** — pin-to-pin only. A restore landing below a losing trial's inflated number, or a losing trial scoring under its baseline, is not a repo regression; firing there invents a degradation that did not happen and names the same event twice. It gets a Gaps line plus a TL;DR mention: either the repo genuinely degraded or the scan is unreliable, and both need a person. Without this the drop lives only in a This-cycle row that vanishes tomorrow, and the Scoreboard keeps an absolute number with no memory that it fell. **Slot order, strictly: (1) never-run cells, (2) restore, (3) stale, (4) top-up.** A never-run cell has an UNKNOWN score and unknown risk; a stale one has a number that is merely old. Deferring a never-run cell for a 27-day-old 74/100 is off-contract, and with no tiebreak among never-run cells an agent invents one differently each run — break ties by catalog order, then project slug. **The cap is 2 refresh slots TOTAL, and a restore consumes one of them** (never a third). 8 cells on a 7-day clock need 1.14 refreshes/day to hold steady; every losing trial mints a restore, so a lab that trials daily and loses often runs at ~1/day and goes BACKWARDS. If the queue is longer than the slots, say so — see the Queue line in §4. **Restore outranks stale and top-up.** A doc corrupted by a LOSING trial is re-scanned on the incumbent NEXT run, in a reserved slot, ahead of stale and top-up. It cannot reach the queue any other way: the overwrite set `last_scanned_at` to today, so the corrupted doc looks the FRESHEST in the project — invisible to the stale rule and to top-up (which takes the oldest). Without the reserved slot the restore instruction is decoration and the wrong score sits there for 7 days. **Fewer than 2 stale?** Top the slot up with the OLDEST committed doc, and say `(top-up, not stale)` in its row. Never leave a slot unfilled with no rule — an agent will invent one, and a caller calling a 3-day-old doc "stale" is how a refresh happens that §1 never authorised. ## §2 One model trial (only against a committed baseline) Run AT MOST ONE trial this run, and ONLY when a **≤7-day committed baseline** exists for the target scorecard (a fresh refresh from §1 counts). No committed baseline within 7 days = no trial (you cannot grade against a score that was never committed) — a named line, not an error. **Which scorecard — pick a (scorecard, candidate) PAIR that can actually pass.** Walk eligible scorecards (≤7-day committed baseline) lowest committed score first, ties → oldest baseline, and take the FIRST whose gate the cheapest non-pinned candidate can meet. Name every scorecard skipped on the way and why. **Re-baseline so a cheap win cannot lock a cell shut.** After promoting a cheaper model, its cost becomes the new ceiling — permanently barring every future candidate on that scorecard. So a scorecard whose pin has been unchanged for 30 days admits ONE candidate up to 1.5× the incumbent's cost, named `rebaseline_probe` in the digest. Cheapness must not become an absorbing state. Eligible-but-not-chosen scorecards are named in the digest, so "why not that one" is never a guess. **A trial RUNS whenever one is eligible** — eligibility is sufficient, not merely permitted; skipping needs a named reason (cap, money stop, paused project, no candidate). Candidate = the CHEAPEST OpenRouter model NOT currently pinned anywhere in the scorecard resolution order, shortlisted from `GET https://openrouter.ai/api/v1/models` **filtered CLIENT-SIDE** (a jq/python one-liner keeping ~8 ids with pricing; apply the Model selection constraints — US-routable provider, the lens's frontmatter bands). **Never curl the full model catalog into context**: it is ~342 models / ~15k chars, and it is then re-sent on every remaining call of the run. **Qualification: projected cost ≤ the last PIN-PRODUCED run's cost on THAT scorecard** (never an unrestored losing trial's cost — that is the lab gating a cell on a number it fabricated itself), computed on the baseline run's real input/output tokens — scans are minutes-long repo jobs, not probes — never a flat dollar ceiling. Over it = named skip, never trialed. OpenRouter unreachable = LOUD RED, no trial. Launch with `input.model: ""` (task → start route → runs route), poll to terminal, apply the same commit check. **Then read back WHICH MODEL ACTUALLY EXECUTED, before grading anything.** The terminal status names it (`GET /api/v1/tasks//status` → the run's `backend.model`). If it is not the challenger, the verdict is the NAMED `challenger_model_not_dispatched`: print both ids side by side, propose nothing, and never grade a number the incumbent produced. **If the status is unreadable or carries no model, that is not a pass** — it is `FAILED scorecard_trial_ungradeable` with the reason `executed_model_unreadable`, same as any other gate you could not evaluate (§2 rule 5). Never retry the trial to get a readable answer: the doc has already been overwritten, so a second run spends a slot and grades the same corrupted baseline. Not hypothetical — between 2026-07-22 and 2026-08-23 the platform overwrote `input.model` with the backend preset model on every run, so EVERY trial in that window silently re-ran the incumbent and would have been graded as the challenger (`docs/plans/2026-08-23-challenger-model-override-never-dispatched.md`; the delivery path is fixed there, this read-back is what makes the next recurrence loud instead of invisible). A trial that cannot say which model produced its number has measured nothing. Grade trial vs baseline: 1. **Hard fails first:** failed run, `scorecard_projection_not_committed`, no completion event, $0/no-output = FAILED; record the code. 2. **Score movement, with a band that has a SOURCE.** Read the baseline's `scorecard_checks` BEFORE launching — no baseline check set = named skip, no trial (cheaper than a wasted run and a corrupted doc). Then: - **|Δ| ≤ 3 points = AGREEMENT.** The entry ticket; go to step 4. - **trial scored HIGHER by >3** = `challenger_scored_higher`. This is a WIN path, not a failure: promote when the per-check diff shows the extra points come from checks the challenger VERIFIED that the incumbent did not. A cheaper model that finds more, with evidence, is the whole point of the lab. - **trial scored LOWER by >3** = `challenger_lost`, incumbent stays. 3. **Evidence quality, with a number so it CAN go red:** FAILED `scorecard_trial_evidence_absent` when **>50% of the scorecard's FULL check set carries `verified: false`**, or when `evidence_label` is empty/boilerplate on more than half the verified ones — regardless of the score. Prose like "overwhelmingly unverified" names no field and no threshold, so it could never fire. 4. **An identical score is AGREEMENT, not a tie** — it is the entry ticket, not the win. It says the cheap model did not disagree; it says nothing about whether it did the work. A "tie" means agreement with NO cost/duration margin, and ties keep the incumbent. **At AGREEMENT, margin to win: ≥25% cheaper OR ≥25% faster** (so persist every §1 scan's duration — without it the "faster" half is permanently unevaluable). A `rebaseline_probe` candidate is allowed up to 1.5× cost, so it can only ever win on DURATION; say so rather than letting it fail a cheapness test it was admitted to bypass. Say so in the digest — "won on cost at score agreement, not on quality" — or the row reads as "the cheap model is as good", which is not what was measured. 5. **A gate you could not evaluate BLOCKS the proposal.** If `scorecard_checks` or the evidence fields are unavailable, the verdict is `FAILED scorecard_trial_ungradeable` — never a proposal. Absence of evidence never authorises an action, and without this the cheapest way to win a trial is to emit a score and no evidence: the one check that would catch it is the one with no data. **A trial's projection OVERWRITES the doc** (score, `last_scanned_at`, and the baseline it was graded against), so after a trial the doc holds a non-pinned model's output and will not look stale for 7 days. Label the row `score is the trial model's, not the pin`; on a LOSS the incumbent is re-run to restore, in the reserved slot. **This lab is REPORT ONLY — it never changes a pin.** `skills_update` is BANNED here (same rule as the reviewer lab): a model switch is a product change, and an agent that both grades the evidence and applies the verdict has no reviewer. It READS pins with `skills_get {skill_id:"project", project_id:""}` (**snake_case `project_id` — camelCase `projectId` is rejected with "project_id is required"**) so the scoreboard shows what is live and so an operator-applied change is noticed. Skills reads are MCP-only — there is no REST skill endpoint for a PAT. Winner with margin → ONE proposal line in the digest's `RECOMMENDING` block: the exact edit (`config.ionwarp.scorecard.model_pins: {: }` in ``'s project.md), the measured margin, both sides' score/cost/duration, and the pre-switch baseline (the pinned model's committed score) so a later regression has something to be judged against. It stands until a run reads the pin and finds it applied — then say so once and reset — or until 14 days pass, reported as expired with its date. A pinned model whose later scans degrade below that baseline gets a revert PROPOSAL, never a revert. ## §3 blocked.json schema `$IONWARP_LAB_STATE_DIR/blocked.json` (see Runtime contracts — resolve it absolutely; a repo-relative path silently finds nothing). Starter `{"blocked": []}` — create it if absent. If it can neither be read nor created, say `blocked_memory_unreadable` in the digest and launch NOTHING that has ever been blocked; never treat an unreadable memory as "nothing blocked". Each entry: `scorecard`, `project`, `error_code`, `since` (UTC date of the MOST RECENT failure), `repeat` (bool — only `true` suppresses a relaunch), `note` (the unblock condition). An entry is live until `since` is >7 days old or the fix ships. **On every repeat: set `repeat: true`, RESET `since` to today, and rewrite `note` to the current unblock condition.** Leaving `since` at the first failure auto-expires the block while the scorecard is still failing daily — the next failure then writes a fresh `repeat:false` entry, which relaunches, so a permanently broken scorecard burns a slot out of a 2-slot budget every 8 days forever. And a stale `note` ("first failure, remembered not blocked") describes the opposite of what the entry now is. ## §4 The digest (≤28 lines, the run's final message) **Delivery: the final message IS the digest** (Runtime contracts above) — first character `🧪`, reason in tool calls never in prose, compose once, stop. This lab's fixed second line: `What this is: repo scorecard refreshes plus at most one cheaper-model trial — report-only.` **Breaker — mirror of the reviewer lab's circuit breaker.** When every scan LAUNCHED in the last 2 runs failed with the same `blocked.json` `error_code` AND no trial is possible (no ready baseline), do not render the full ceremony: emit a ≤5-line RED digest — what is blocked, its code, since when, and who unblocks it — and stop. Re-printing an unchanged scoreboard around a wall of blocked lines is how two identical 27-line digests once shipped on consecutive days while measuring nothing. ONE final message. Over budget, cut `Queue` and `Gaps` detail before a measurement line, a recommendation, or a link — those three ARE the message. ≤28 lines. **Formatting — same convention as the reviewer lab.** The common delivery surface renders Slack mrkdwn: `**bold**` (DOUBLE asterisks — a bare `*x*` often renders as _italic_), `` ``` `` code fences (monospace, for aligning columns), and `` links — but NOT inside a fence. So: **bold + emoji on every section header** (scannable), the **Scoreboard and This-cycle TABLES inside a ` ``` ` fence** (columns align), the **TRIAL's two measurement rows inside their own fence**, and every **link on a normal mrkdwn line outside any fence** (or it renders literally). Emoji legend, one per line: 📋 scoreboard · 🔄 this cycle · 🧪 trial · 💡 recommending · 🔗 read · 🚧 blocked · 📥 queue · ⚠️ gaps · ✅ won · ❌ failed · ➖ agreement/no-change. Never wrap the whole digest in one fence — it kills every link and bold. The blocks, in order: - **TL;DR** — 1–2 plain sentences: how many scanned / committed / failed, what was trialed and the verdict, TOTAL SPEND, and anything WAITING ON A HUMAN. If nothing needs a person, say "nothing needs you". - **Scoreboard — EVERY catalog scorecard × project, not only the ones touched.** `project · scorecard · model · score/100 · age · open actions`. `model` is the RESOLVED PIN, read from the ONE canonical place it already lives: the scorecard skill's own frontmatter `execution.model` (the card front), overridden by `project.md` `ionwarp.scorecard.model_pins`. Never store a model anywhere else and never re-derive it from `last_scan_task_id` — that is a second source of truth AND 8 extra task fetches a run. Flag `(trial)` when the committed score came from an unpinned trial model. A trial overwrite must CARRY FORWARD the pinned baseline's `action_item_count` rather than null it. Show `?` only if the baseline value is genuinely unknown, with the restore pending. `open actions` is the doc's `action_item_count`: it is the only field that says what to FIX — read it on every doc, never discard it. Higher score = better; say so once in the legend. **The score cell has THREE states, and `status` decides which — never the presence of a `score` field.** Reading `score` without reading `status` is what once printed two failed scans as bare current scores under a TL;DR that said both had FAILED: 1. `status == "ready"` → `/` plus its age. This is the only state that may render a bare number. 2. doc exists but `status` is anything else (`running`, `failed`) → render **`/ (stale — last scan )`**, or `— (never committed)` when no score was ever committed. The number is a PREVIOUS committed result that the product deliberately preserves (a starting scan writes a merge-only `{status:"running"}` and never zeroes a committed score), so it is real history — but it is NOT this cycle's result and must never be shown as though it were current. 3. no doc for a catalog id → `—` (never run). The qualifier rides **in the cell**, not in a Gaps line 20 rows below it: a caveat the reader reaches after they have already believed the number is decoration. Do not "fix" this by dropping the number — the history is worth showing; show it labelled. Age comes from `last_scanned_at`. When that field is null the age cell is `—`, never a number derived from `reportedAt`/`updatedAt` (those move on the merge-only start write, so they measure when a scan LAUNCHED, not when a score was COMMITTED — that is how a failed scan can look freshly scored). - **This cycle** — one row per scan actually launched: `project · scorecard · score old→new (or first score) · outcome · cost · duration · link`. **Duration on EVERY row, not just the trial line.** §2 already requires persisting every scan's duration for the ≥25%-faster margin, and a cycle row without it cannot answer "why did this take 25 minutes" — the same gap that once made the reviewer digests undiagnosable. `committed` means the projection was READ BACK from the doc; write `committed (read back)` the first time it appears. - **TRIAL — the same two-line measurement shape the reviewer lab uses**, because it is the same question asked of a different work unit: challenger and pinned on their own lines, SAME fields in the SAME column order, so the comparison is read vertically and never computed by the reader. Every field comes from that scan's own telemetry; a field you did not measure prints `—` and names itself in Gaps (`latency_unmeasured`, `tps_unreported`), never a blank that reads as a zero: Rendered (fence the two measurement rows; `checks`/`why`/`read` are mrkdwn below it so the link stays clickable). `V/C` = checks verified / total · `$/Mtok-in` = cost ÷ (input + cache_read + cache_creation) · higher score = better: ➖ *ai-readiness* · `` — kimi-k3 vs pinned deepseek-v4-pro-0813 → *agreement, won on speed* ``` challenger 72/100 · V14/18 · 9 actions · $0.041 · 612s · 3,140 out tok · 0.33 $/Mtok-in pinned 74/100 · V16/18 · 9 actions · $0.088 · 1504s · 4,002 out tok · 1.81 $/Mtok-in! (baseline ) ``` checks · the 2 points are checks the pin verified and kimi did not (secret-scanning, dependency-pinning); no check moved the other way why · |Δ|2 ≤ 3 = agreement, then *53% cheaper* and *59% faster* — both margins met read · <…/tasks/ts_x|trial scan> · <…/scorecards/ai-readiness|scorecard> - `score/100` with the Δ against the committed baseline — the verdict axis (§2 step 2), and `higher = better` said once in the legend. - `V/` — checks `verified: true` over the FULL check set. This is the evidence gate (§2 step 3) made visible: a score with half its checks unverified is not the same result as the same score fully verified, and printing only the score hides exactly that. - `actions` — `action_item_count`, the only field that says what to FIX. - `cost`, `duration`, `out tok`, `$/Mtok-in` — same definitions and the same always-printed rule as the reviewer lab, which prints the scope-normalized `$/Mtok-in` where a naive digest would print tok/s. Follow it: a tok/s figure derived from `output_tokens` excludes reasoning tokens on every reasoning model pinned here, so it is fiction. The `!` band marker comes from the skill's own frontmatter `model_bands` (Runtime contracts) — this skill re-states no number of its own. **Latency is always printed**: it is half the promotion margin here (≥25% cheaper OR ≥25% faster), so omitting it makes a win unauditable. - `(baseline )` on the pinned line when its numbers come from the committed baseline scan rather than a fresh launch — the scorecard equivalent of the reviewer lab's `(reused)`. - The `checks` line names WHICH checks moved, not just how many points. A score delta with no named check is an opinion. No trial ran → the named reason, or `eligible, not run — `. Name the eligible scorecards NOT chosen either way. - **RECOMMENDING** — the pin proposal (§2) in full when a trial cleared the bar, or a standing one re-printed with its age, or a proposal reported applied / expired once. Say `No change today.` when there is none. End a proposal with who acts: "I don't change pins — this one is yours to apply." - **Read** — links, `` format, every URL built from an id read THIS run: a scorecard = `$IONWARP_API_URL//scorecards/`, a scan = `$IONWARP_API_URL//tasks/`, a post = `$IONWARP_API_URL/blog/`. Every scan and trial in `This cycle` carries its link — the score exists so someone can go read what the model wrote, and a digest without links makes that a hunt. A missing id is the named gap `task_url_unavailable ()`, never a guessed URL. - **Blocked** — entries active this run, AND entries CLEARED this run with why (a blocker dying is the most useful thing that can happen to this list; a bare `none` hides it). Omit when both are empty. - **Queue** — cells owed, slots available, earliest clear date. Backlog is otherwise visible only as an ABSENCE, and absences are exactly what nobody reads; this is the line that shows a newly-unpaused project starving on the day it starts rather than three runs later. - **Gaps** — failed scans, deferrals, unreachable sources, paused projects. A paused project gets ONE plain sentence naming what is waiting. Task ids go in the appendix, never the main message. **A failed scan's error CODE goes in its row** — a failure row without its code is useless; only ids and stack detail belong in the appendix. ## Verify (self-assert) Checkable at emit time and not obvious from the body: (1) ≤2 refresh slots and ≤1 trial, task ids in the appendix only; (2) every scan either passed the commit check or renders its NAMED failure — never a score, never a 0; (3) no `skills_update` was called and no pin was written — a cleared bar is a PROPOSAL carrying its margin, both sides' numbers, and the pre-switch baseline; (4) no `projects_update`, no `run_mode` change, no PAT printed; (5) every scan, trial, and scorecard named in the digest carries its link or its named `task_url_unavailable` gap; (6) a trial that ran carries TWO measurement lines with the same fields in the same order — score/100, V/C checks, actions, cost, duration, out tok, $/Mtok-in — plus the `checks` line naming which checks moved. No field is blank: an unmeasured axis is `—` plus a named Gap. (7) The final message is the digest and nothing else — `🧪` first character.