config-drift-checker · komo-stack · mutation test
12 stayed green under a case that claims them; with no plugin installed the model already does what 7 of them test.
Each mutant is one small sabotage of a temp copy of the setup: a deleted or inverted rule, a deleted section, a blurred skill trigger, a removed skill or a disabled hook. A good suite goes red on every one.
A case claims them, but it stayed green with the sabotage in place. Tighten its graders.
Their graders pass even with no plugin installed, so today these rules cost context without changing behaviour. Keep the graders either way: on every Claude Code or model release they tell you if the model stops doing it by default. Graders are matched to rules by the words they share, so check that the named grader really tests the rule before you drop it.
| result | operator | mutant | caught by | cost |
|---|---|---|---|---|
| missed | delete-rule | deleted rule 'Lombok is not on the classpath. Never use it (@RequiredArgs…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/lombok-is-not-on-the-classpath the model does this with no plugin (grader no-field-injection passes 1 of 1 runs without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-controller-follows-conventions 1.00 to 1.00 | $0.11 |
| missed | invert-rule heuristic | inverted rule 'Lombok is not on the classpath. Never use it (@RequiredArgs…' in skills/spring-boot-conventions/SKILL.md ('Never' became 'Always') invert-rule:skill/spring-boot-conventions/lombok-is-not-on-the-classpath the model does this with no plugin (grader no-field-injection passes 1 of 1 runs without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-controller-follows-conventions 1.00 to 1.00 | $0.11 |
| caught | delete-rule | deleted rule 'Request and response shapes are Java records, grouped in a…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/request-and-response-shapes-are-java | spring-request-dtos-are-validated-records 1.00 to 0.67 | $0.11 |
| missed | delete-rule | deleted rule 'Validate request bodies with jakarta.validation (@Valid, @N…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/validate-request-bodies-with-jakarta-validation the model does this with no plugin (graders constraint-annotations and valid-annotation pass every run without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-request-dtos-are-validated-records 1.00 to 1.00 | $0.11 |
| missed | delete-rule | deleted rule '@RestController + @RequestMapping("/api/<area>"). Thin: par…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/restcontroller-requestmapping-api-area-thin-parse | spring-controller-follows-conventions 1.00 to 0.86 | $0.11 |
| missed | delete-rule | deleted rule 'Every response is wrapped: ApiResponse.ok(body) or ApiRespo…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/every-response-is-wrapped-apiresponse-ok | spring-controller-follows-conventions 1.00 to 0.86 | $0.11 |
| missed | invert-rule heuristic | inverted rule 'Every response is wrapped: ApiResponse.ok(body) or ApiRespo…' in skills/spring-boot-conventions/SKILL.md ('Never' became 'Always') invert-rule:skill/spring-boot-conventions/every-response-is-wrapped-apiresponse-ok the swap is in a later sentence, so the rest of the rule still says the opposite; the model may follow that; 2 other rules in section 'Controllers' still stand and may say the opposite; the delete-section mutant tests them together | spring-controller-follows-conventions 1.00 to 1.00 | $0.11 |
| caught | delete-rule | deleted rule 'Pagination is 1-based page and size query params. Clamp: in…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/pagination-is-1-based-page | spring-controller-follows-conventions 1.00 to 0.71 | $0.10 |
| missed | delete-rule | deleted rule 'Business rules and authorization checks live in the service…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/business-rules-and-authorization-checks-live the model does this with no plugin (grader controller-and-service passes 1 of 1 runs without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-service-owns-rules-and-errors 1.00 to 1.00 | $0.16 |
| missed | delete-rule | deleted rule 'Services return records or small result types, never Map<St…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/services-return-records-or-small-result the model does this with no plugin (grader no-map-result passes 1 of 1 runs without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-service-owns-rules-and-errors 1.00 to 1.00 | $0.14 |
| missed | invert-rule heuristic | inverted rule 'Services return records or small result types, never Map<St…' in skills/spring-boot-conventions/SKILL.md ('never' became 'always') invert-rule:skill/spring-boot-conventions/services-return-records-or-small-result 1 other rule in section 'Services' still stands and may say the opposite; the delete-section mutant tests them together; the model does this with no plugin (grader no-map-result passes 1 of 1 runs without it), so today the rule is not steering anything. You can drop it to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put it back | spring-service-owns-rules-and-errors 1.00 to 0.86 | $0.17 |
| missed | delete-rule | deleted rule 'Throw domain exceptions; a @RestControllerAdvice handler ma…' from skills/spring-boot-conventions/SKILL.md delete-rule:skill/spring-boot-conventions/throw-domain-exceptions-a-restcontrolleradvice-handler | spring-service-owns-rules-and-errors 1.00 to 0.86 | $0.16 |
| missed | invert-rule heuristic | inverted rule 'Throw domain exceptions; a @RestControllerAdvice handler ma…' in skills/spring-boot-conventions/SKILL.md ('Do not' became 'Do') invert-rule:skill/spring-boot-conventions/throw-domain-exceptions-a-restcontrolleradvice-handler the swap is in a later sentence, so the rest of the rule still says the opposite; the model may follow that | spring-service-owns-rules-and-errors 1.00 to 1.00 | $0.14 |
| caught | delete-section | deleted section 'DTOs' (2 rules) from skills/spring-boot-conventions/SKILL.md delete-section:skills/spring-boot-conventions/SKILL.md#dtos | spring-request-dtos-are-validated-records 1.00 to 0.67 | $0.12 |
| caught | delete-section | deleted section 'Controllers' (3 rules) from skills/spring-boot-conventions/SKILL.md delete-section:skills/spring-boot-conventions/SKILL.md#controllers | spring-controller-follows-conventions 1.00 to 0.43 | $0.11 |
| missed | delete-section | deleted section 'Services' (2 rules) from skills/spring-boot-conventions/SKILL.md delete-section:skills/spring-boot-conventions/SKILL.md#services the model does this with no plugin (graders controller-and-service and no-map-result pass every run without it), so today these rules are not steering anything. You can drop them to save context, but keep the grader: it becomes a tripwire, and the release canary turns red if a new model stops doing this, which is your cue to put them back | spring-service-owns-rules-and-errors 1.00 to 1.00 | $0.14 |
| caught | blur-trigger | blurred the trigger description of skill 'spring-boot-conventions' in skills/spring-boot-conventions/SKILL.md blur-trigger:skill/spring-boot-conventions | spring-request-dtos-are-validated-records 1.00 to 0.83 | $0.58 |
| caught | drop-skill | removed skill 'spring-boot-conventions' (skills/spring-boot-conventions/) entirely drop-skill:skill/spring-boot-conventions | spring-controller-follows-conventions 1.00 to 0.43 | $0.59 |
| caught | disable-hook | disabled hook PreToolUse on Bash (node ${CLAUDE_PLUGIN_ROOT}/hooks-handlers/guard.m…) in hooks/hooks.json disable-hook:hook/pretooluse-bash | guard-blocks-destructive-git 1.00 to 0.50 | $0.08 |
Scores are the mean over a case's runs with the setup loaded, against a reference run of the unmutated setup. Inverted rules are a word swap and can read oddly, so they are marked heuristic. Generated by config-drift-checker mutate.