What this report checks · 72 grader verdicts across 18 agent runs on 6 cases. A plain claude plugin eval run stops there; the chips are this tool's layer on top:
✓ baseline diff✓ noise bands✓ escalation guards· refusal screening· discovered vs invoked✓ efficiency drift✓ setup health✓ suite completeness
Setup health on Claude Code 2.1.295 · skill frontmatter and eval suite format, checked before the run; these checks start no model runs.
Eval suite format✓ cleanloaded 6 of 6 cases · confirmed by the runner
✓ No setup problems: 1 skill and 6 eval cases pass every static check, and the runner loaded the suite.
How to read this report, and what to do
No drift / baseline recorded: nothing to do. Hover a grader chip to see what each check asserts and why it passed.
N case(s) regressed: open the red case(s) and classify each failing run: refused or asked before acting (1 turn, no tool calls) → the case never reached the skill/hook, rewrite the scenario; skill/hook did not fire → a real regression: pin model.pinned/harness.pinned in .cdc.yml to the last good pair, fix the setup (or run the repair skill), tell the maintainers; grader wrong (matched prose, a negation, nested parentheses) → fix the grader and re-score with --regrade; flaky (mixed verdicts across runs) → raise runs, never the threshold.
Efficiency drift (slower / pricier / longer): every case still passes, but the median turns, cost or time moved past its threshold. Warning by default; add it to fail_on in .cdc.yml to make it red.
Noisy (⚠): the case dropped past the threshold but stayed within its historical noise band, a warning, not a regression; more runs per case shrink the noise band.
Runs errored: read the error text, usually no prepaid API credit or a Claude Code startup failure. Nothing was stored; fix and re-run.
A run shows max_turns: it was cut short and scored as-is (amber) → raise that case's max_turns.
You changed the setup on purpose: re-run with promote-baseline: true so this becomes the new baseline.
⚠ baseline quality (never red):spring-service-owns-rules-and-errors, unstable baseline (±0.20). More runs per case fix this; never loosen the threshold.
Scores are the mean over a case's runs with the setup loaded; a drop of more than 0.15 against the baseline is a regression, a drop inside the case's noise band is a ⚠ warning, not red. Indicators (ind) are recorded but not scored. Generated by config-drift-checker.