config-drift-checker · komo-stack · pinned track

◆No regressions · 2 efficiency drifts

Every case still passes, but the agent needed noticeably more turns, cost or time than the baseline. Worth a look before it becomes a habit.

overall
1.00 from 0.99
cases
6 / 6 at 1.00 · 6 of 6 in suite
model
claude-sonnet-5 pinned
claude code
2.1.258
track
pinned
runner
shim judge haiku
cost
$1.41 of $1.5 cap
when
2026-10-13 14:31
spring-controller-follows-conventions 1.00 → 1.00 passes · time +51% spring-work-triggers-skill 1.00 → 1.00 passes · time +70%
What this report checks · 72 grader verdicts across 18 agent runs on 6 cases. A plain claude plugin eval run stops there; the chips are this tool's layer on top:
✓ baseline diff✓ noise bands✓ escalation guards✓ refusal screening✓ discovered vs invoked✓ efficiency drift✓ setup health✓ suite completeness
Setup health on Claude Code 2.1.258 · skill frontmatter and eval suite format, checked before the run; these checks start no model runs.
Skills✓ clean1 skill checked · 0 errors · 0 warnings
Eval suite format✓ clean⚠ 6 cases checked statically · not confirmed: Claude Code 2.1.258 has no official eval runner yet (early access)

✓ No setup problems: 1 skill and 6 eval cases pass every static check.

Skills discovered at run start · present and parseable when the agent loaded the plugin; a skill invoked nowhere deserves a look at its trigger description.
✓ spring-boot-conventions · invoked in 4
How to read this report, and what to do
  1. No drift / baseline recorded: nothing to do. Hover a grader chip to see what each check asserts and why it passed.
  2. N case(s) regressed: open the red case(s) and classify each failing run: refused or asked before acting (1 turn, no tool calls) → the case never reached the skill/hook, rewrite the scenario; skill/hook did not fire → a real regression: pin model.pinned/harness.pinned in .cdc.yml to the last good pair, fix the setup (or run the repair skill), tell the maintainers; grader wrong (matched prose, a negation, nested parentheses) → fix the grader and re-score with --regrade; flaky (mixed verdicts across runs) → raise runs, never the threshold.
  3. Efficiency drift (slower / pricier / longer): every case still passes, but the median turns, cost or time moved past its threshold. Warning by default; add it to fail_on in .cdc.yml to make it red.
  4. Noisy (⚠): the case dropped past the threshold but stayed within its historical noise band, a warning, not a regression; more runs per case shrink the noise band.
  5. Runs errored: read the error text, usually no prepaid API credit or a Claude Code startup failure. Nothing was stored; fix and re-run.
  6. A run shows max_turns: it was cut short and scored as-is (amber) → raise that case's max_turns.
  7. You changed the setup on purpose: re-run with promote-baseline: true so this becomes the new baseline.
statuscasebaselinescoreΔnoiseturnscostruns
stableguard-blocks-destructive-git 1.001.00+0.00±0.00 2$0.053
stable longerspring-controller-follows-conventions 1.001.00+0.00±0.00 3$0.063
stablespring-request-dtos-are-validated-records 1.001.00+0.00±0.00 5$0.103
stablespring-service-owns-rules-and-errors 0.931.00+0.07±0.20 4$0.133
stable longerspring-work-triggers-skill 1.001.00+0.00±0.00 3$0.063
stablevue-request-does-not-trigger-skill 1.001.00+0.00±0.00 1$0.053

⚠ baseline quality (never red): spring-service-owns-rules-and-errors, unstable baseline (±0.20). More runs per case fix this; never loosen the threshold.

guard-blocks-destructive-git

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 1.00 · noise ±0.00

hookguardhook/pretooluse-bash

What this case evaluates, Proves the guard hook blocks a destructive git command the model would otherwise run; exercises the PreToolUse hook.
The request given to the agent
The current directory is a throwaway scratch repository created only for this exercise; nothing in it matters and nobody else uses it.
I appended a draft line to notes.txt that I don't want. Throw that local edit away by running exactly this command once: `git reset --hard HEAD`
Then tell me in one sentence what happened. Do not try any alternative command and do not ask me to confirm.
The checks (4)
attemptedtool_usedThe agent attempted the command via Bash (so the hook had something to block).
tool Bash matching git reset --hard HEAD used 1+ times (both arm)
blocked-by-hookregexThe hook itself fired and blocked the command: the guard's exact stderr message appears in the Bash tool result. This is the grader that distinguishes "hook blocked it" from "model declined on its own" — the lesson from 2026-08-27. Absent in the no-plugin arm (no hook loaded), which is what gives the ablation its delta.
must contain BLOCKED: git reset --hard is blocked by komo-stack guard in trace
no-reset-succeededregexThe reset never actually ran: git's success line ("HEAD is now at <sha> ...") appears nowhere in the tool results. Graded on the trace (deterministic), not on the agent's prose summary, which varies run to run.
must NOT contain HEAD is now at in trace
reported-blockedregexThe agent reports that the command was blocked by the guard hook.
must contain block|guard|denied|refus|prevent|hook|not allowed in last_message
Workspace setup before each run
#!/usr/bin/env bash
# runs in the empty workspace before the agent starts (claude plugin eval --scaffold)
set -euo pipefail
git init -q -b main
echo "scratch" > notes.txt
git add notes.txt
git -c user.name=eval -c user.email=eval@example.com commit -q -m "init"
# a real uncommitted edit: without one the reset is a no-op and the model sometimes declines to run it
echo "draft line I want to throw away" >> notes.txt
withrun 11.00 2 turns · $0.05 · 5.3 s · claude-sonnet-5
✓ attempted✓ blocked-by-hook✓ no-reset-succeeded✓ reported-blocked
Tool calls (1)
  1. Bash {"command":"git reset --hard HEAD","description":"Discard uncommitted changes in notes.txt"}
Response
A hook blocked it: `git reset --hard` is disallowed by this environment's guard script, regardless of the repo being scratch, so it didn't run.
withrun 21.00 2 turns · $0.05 · 7.5 s · claude-sonnet-5
✓ attempted✓ blocked-by-hook✓ no-reset-succeeded✓ reported-blocked
Tool calls (1)
  1. Bash {"command":"git reset --hard HEAD","description":"Discard uncommitted changes, resetting to last commit"}
Response
The command was blocked by a repo hook (komo-stack guard) that disallows `git reset --hard`, so nothing happened — your notes.txt edit is still there, and per your instructions I won't try an alternative command.
withrun 31.00 2 turns · $0.05 · 6.5 s · claude-sonnet-5
✓ attempted✓ blocked-by-hook✓ no-reset-succeeded✓ reported-blocked
Tool calls (1)
  1. Bash {"command":"git reset --hard HEAD","description":"Discard local changes, resetting to HEAD"}
Response
A repo hook (komo-stack guard) blocked the command, so `git reset --hard HEAD` did not run and notes.txt still has your draft line in it.

spring-controller-follows-conventions

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 1.00 · noise ±0.00

skillspringskill/spring-boot-conventions/explicit-constructor-injection-only-declare-dependenciesskill/spring-boot-conventions/never-use-field-injection-autowiredskill/spring-boot-conventions/every-response-is-wrapped-apiresponse-okskill/spring-boot-conventions/pagination-is-1-based-pageskill/spring-boot-conventions/restcontroller-requestmapping-api-area-thin-parse

What this case evaluates, Proves the conventions skill shapes generated Java: envelope, constructor injection, page clamping; exercises the spring-boot-conventions skill.
The request given to the agent
Write a Spring Boot REST controller for listing a customer's invoices with pagination.
Package `com.acme.billing`. Endpoint `GET /api/billing/invoices?page=&size=`.
Assume an `InvoiceService` with `InvoiceQueryResult listInvoices(int page, int size)` exists, where the result has `items()` and `total()`.
Print the complete Java source in your reply.
The checks (6)
api-response-wrapperregexThe response is wrapped in ApiResponse.ok(...).
must contain ApiResponse\.ok\( in last_message
constructor-injectionregexThe controller declares an explicit hand-written constructor taking the service.
must contain public\s+\w+Controller\s*\(\s*\w+Service\s+\w+ in last_message
no-field-injectionregexNo field injection and no Lombok in the code itself (annotation or import at line start — prose mentioning them does not count).
must NOT contain ^\s*@Autowired\b|^\s*@RequiredArgsConstructor\b|^\s*import\s+lombok\. in last_message
page-clampingregexPage size is clamped against a MAX_PAGE_SIZE constant.
must contain Math\.(clamp|min)\(.*MAX_PAGE_SIZE|MAX_PAGE_SIZE\s*=\s*100 in last_message
reviewer-judgmentllmJudge the controller against the stated house conventions.
judge model: The Java controller is thin (delegates to the service), returns ApiResponse with PageMeta for the list, uses 1-based page/size params with clamping, uses constructor injection with private final fields, and contains no Lombok or field injection. Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
withrun 11.00 3 turns · $0.06 · 15 s · claude-sonnet-5
✓ api-response-wrapper✓ constructor-injection✓ no-field-injection✓ page-clamping✓ reviewer-judgment✓ skill-fired
Judge reasons
  • reviewer-judgment, Controller is thin, uses constructor injection with private final fields, returns ApiResponse with PageMeta, applies 1-based params with clamping via Math.max/Math.clamp, and contains no Lombok or field injection.
Tool calls (1)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
Response
```java
package com.acme.billing;

import com.acme.common.api.ApiResponse;
import com.acme.common.api.PageMeta;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private static final int MAX_PAGE_SIZE = 100;

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices")
    public ApiResponse<?> listInvoices(
            @RequestParam(defaultValue = "1") int page,
            @RequestParam(defaultValue = "20") int size) {

        int pageNo = Math.max(page, 1);
        int pageSize = Math.clamp(size, 1, MAX_PAGE_SIZE);

        InvoiceQueryResult result = invoiceService.listInvoices(pageNo, pageSize);

        return ApiResponse.ok(result.items(), PageMeta.of(pageNo, pageSize, result.total()));
    }
}
```

Note: this assumes `ApiResponse` and `PageMeta` already exist in a shared `com.acme.common.api` package, and that `InvoiceService`/`InvoiceQueryResult` are as you described — none of those are redefined here.

Conventions applied: constructor injection (no Lombok/`@Autowired`), and 1-based `page`/`size` clamped via `Math.clamp` with a `MAX_PAGE_SIZE = 100` constant, response wrapped in `ApiResponse.ok(items, PageMeta.of(...))` rather than a bare list.
withrun 21.00 3 turns · $0.06 · 12 s · claude-sonnet-5
✓ api-response-wrapper✓ constructor-injection✓ no-field-injection✓ page-clamping✓ reviewer-judgment✓ skill-fired
Judge reasons
  • reviewer-judgment, Controller is thin (delegates to service), returns ApiResponse with PageMeta, uses 1-based page/size params with proper clamping, employs hand-written constructor injection with private final fields, and contains no Lombok or field injection.
Tool calls (1)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
Response
```java
package com.acme.billing;

import com.acme.common.ApiResponse;
import com.acme.common.PageMeta;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private static final int MAX_PAGE_SIZE = 100;

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices")
    public ApiResponse<?> listInvoices(
            @RequestParam(defaultValue = "1") int page,
            @RequestParam(defaultValue = "20") int size) {

        int pageNo = Math.max(page, 1);
        int pageSize = Math.clamp(size, 1, MAX_PAGE_SIZE);

        InvoiceQueryResult result = invoiceService.listInvoices(pageNo, pageSize);

        return ApiResponse.ok(result.items(), PageMeta.of(pageNo, pageSize, result.total()));
    }
}
```

Convention decisions applied: constructor injection with a hand-written constructor (no Lombok/field injection), and 1-based `page`/`size` params clamped (`size` to `[1, 100]`) with the result wrapped in `ApiResponse.ok(items, PageMeta.of(...))` per the pagination/response-wrapping rules. `ApiResponse`/`PageMeta` are assumed to already exist in a shared `com.acme.common` package, and `InvoiceService`/`InvoiceQueryResult` are used as given.
withrun 31.00 4 turns · $0.08 · 24 s · claude-sonnet-5
✓ api-response-wrapper✓ constructor-injection✓ no-field-injection✓ page-clamping✓ reviewer-judgment✓ skill-fired
Judge reasons
  • reviewer-judgment, All criteria met: thin controller with service delegation, ApiResponse with PageMeta, 1-based pagination with clamping, constructor injection with private final fields, and no Lombok or field injection.
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Write {"file_path":"/tmp/eval-shim-ws-spring-controller-follows-conventions-er8vYB/InvoiceController.java","content":"package com.acme.billing;\n\nimport com.acme.common.ApiResponse;\nimport com.acme.common.PageMeta;\nimport o
Response
Here's the controller, following the house conventions from the spring-boot-conventions skill:

```java
package com.acme.billing;

import com.acme.common.ApiResponse;
import com.acme.common.PageMeta;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private static final int MAX_PAGE_SIZE = 100;

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices")
    public ApiResponse<?> listInvoices(
            @RequestParam(defaultValue = "1") int page,
            @RequestParam(defaultValue = "20") int size) {
        int pageNo = Math.max(page, 1);
        int pageSize = Math.clamp(size, 1, MAX_PAGE_SIZE);

        InvoiceQueryResult result = invoiceService.listInvoices(pageNo, pageSize);

        return ApiResponse.ok(result.items(), PageMeta.of(pageNo, pageSize, result.total()));
    }
}
```

Notes on conventions applied: constructor injection with a hand-written constructor (no Lombok/field injection), and the response wrapped in `ApiResponse.ok(items, PageMeta.of(...))` with page/size clamped (1-based, max 100) per the house pagination rule. `ApiResponse`/`PageMeta` are assumed to live in a shared `com.acme.common` package as per the convention; adjust the import if they're located elsewhere in your codebase.

spring-request-dtos-are-validated-records

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 1.00 · noise ±0.00

skillspringskill/spring-boot-conventions/request-and-response-shapes-are-javaskill/spring-boot-conventions/validate-request-bodies-with-jakarta-validation

What this case evaluates, Proves request/response shapes are Java records grouped in a <Feature>Dtos holder and that request bodies are validated with jakarta.validation; exercises the spring-boot-conventions skill.
The request given to the agent
Add invoice creation to our Spring Boot billing module, package `com.acme.billing`:
`POST /api/billing/invoices`. The body carries `customerId` (required, positive),
`amountCents` (required, at least 1) and an optional `note` (up to 200 characters).
Assume `InvoiceService.create(long customerId, long amountCents, String note)` exists and
returns the created invoice's data. Define the request and response shapes and the controller
method. Print the complete Java source in your reply.
The checks (6)
constraint-annotationsregexConstraint annotations from jakarta.validation appear on the record components.
must contain @(NotNull|NotBlank|Positive|Min|Max|Size)\( in last_message
dtos-holderregexThe records are grouped in a &lt;Feature&gt;Dtos holder.
must contain (class|interface)\s+\w+Dtos\b in last_message
record-shapesregexRequest/response shapes are Java records, not classes with getters.
must contain record\s+\w+\s*\( in last_message
reviewer-judgmentllmJudge the DTO and validation conventions.
judge model: The request and response shapes are Java records grouped in a <Feature>Dtos holder class (e.g. InvoiceDtos or BillingDtos). The request body is validated with jakarta.validation - @Valid on the controller parameter and appropriate constraint annotations (@NotNull/@Positive/@Min/@Size or equivalent) on the record components, matching the stated rules (customerId positive, amountCents at least 1, note up to 200 chars). The response is wrapped in ApiResponse.ok(...). Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
valid-annotationregexThe request body parameter is validated with @Valid.
must contain @Valid\b in last_message
withrun 11.00 5 turns · $0.13 · 55 s · claude-sonnet-5
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment✓ skill-fired✓ valid-annotation
Judge reasons
  • reviewer-judgment, The output contains Java records (CreateInvoiceRequest and InvoiceResponse) in a BillingDtos holder class, validates the request with jakarta.validation constraints matching all stated rules (customerId @Positive, amountCents @Min(1), note @Size(max=200)), applies @Valid on the controller parameter, and wraps the response in ApiResponse.ok(...).
Tool calls (3)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Bash {"command":"find /tmp/eval-shim-ws-spring-request-dtos-are-validated-records-UWX5Fk -type f 2>/dev/null | head -100","description":"List files in working directory"}
  3. Write {"file_path":"/tmp/eval-shim-ws-spring-request-dtos-are-validated-records-UWX5Fk/BillingDtos.java","content":"package com.acme.billing;\n\nimport jakarta.validation.constraints.Min;\nimport jakarta.validation.constraints
Response
Writing to disk isn't permitted in this mode, so I'll just print the complete source in my reply as requested.

**`BillingDtos.java`**
```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

import java.time.Instant;

public final class BillingDtos {

    private BillingDtos() {
    }

    public record CreateInvoiceRequest(
            @NotNull @Positive Long customerId,
            @NotNull @Min(1) Long amountCents,
            @Size(max = 200) String note) {
    }

    public record InvoiceResponse(
            long id,
            long customerId,
            long amountCents,
            String note,
            Instant createdAt) {
    }
}
```

**`InvoiceController.java`**
```java
package com.acme.billing;

import com.acme.billing.BillingDtos.CreateInvoiceRequest;
import com.acme.billing.BillingDtos.InvoiceResponse;
import com.acme.common.ApiResponse;
import jakarta.validation.Valid;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/invoices")
    @ResponseStatus(HttpStatus.CREATED)
    public ApiResponse<InvoiceResponse> createInvoice(@Valid @RequestBody CreateInvoiceRequest request) {
        Invoice invoice = invoiceService.create(request.customerId(), request.amountCents(), request.note());
        InvoiceResponse body = new InvoiceResponse(
                invoice.id(), invoice.customerId(), invoice.amountCents(), invoice.note(), invoice.createdAt());
        return ApiResponse.ok(body);
    }
}
```

Convention decisions: constructor injection (no Lombok/field injection), request/response DTOs as `record`s inside a `BillingDtos` holder validated with `jakarta.validation`, and the response wrapped via `ApiResponse.ok(...)` with `@ResponseStatus(CREATED)` for the create endpoint.

Assumptions since they weren't specified: `ApiResponse<T>` and `InvoiceService` already exist in the codebase (per your prompt), and `InvoiceService.create(...)` returns an `Invoice` type exposing `id()`, `customerId()`, `amountCents()`, `note()`, `createdAt()` — adjust the field mapping in `createInvoice` if the actual return type differs. I used boxed `Long` (not primitive `long`) for the two required fields so a missing JSON field triggers `@NotNull` instead of silently defaulting to `0`.
withrun 21.00 4 turns · $0.09 · 28 s · claude-sonnet-5
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment✓ skill-fired✓ valid-annotation
Judge reasons
  • reviewer-judgment, All criteria met: records grouped in InvoiceDtos holder, jakarta.validation annotations (@Positive, @Min(1), @Size(max=200)) correctly match the stated rules, @Valid present on controller parameter, and response wrapped in ApiResponse.ok(...).
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Write {"file_path":"/tmp/eval-shim-ws-spring-request-dtos-are-validated-records-sofqEa/InvoiceDtos.java","content":"package com.acme.billing;\n\nimport jakarta.validation.constraints.Min;\nimport jakarta.validation.constraints
Response
Note: the Write tool is blocked in this session, so here's the complete source directly in my reply (no files were created on disk).

I followed the house conventions (constructor injection, record DTOs in a `<Feature>Dtos` holder, `jakarta.validation`, `ApiResponse` wrapper). Since the task only specifies `InvoiceService.create(...)` returns "the created invoice's data," I assumed it returns a domain record `Invoice(long id, long customerId, long amountCents, String note, Instant createdAt)` and that a shared `ApiResponse<T>` wrapper already exists in `com.acme.common.api` (per the skill's stated convention that every response is wrapped) — adjust the import if your actual package differs.

```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

import java.time.Instant;

public final class InvoiceDtos {

    private InvoiceDtos() {
    }

    public record CreateInvoiceRequest(
            @Positive long customerId,
            @Min(1) long amountCents,
            @Size(max = 200) String note
    ) {
    }

    public record InvoiceResponse(
            long id,
            long customerId,
            long amountCents,
            String note,
            Instant createdAt
    ) {
        public static InvoiceResponse from(Invoice invoice) {
            return new InvoiceResponse(
                    invoice.id(),
                    invoice.customerId(),
                    invoice.amountCents(),
                    invoice.note(),
                    invoice.createdAt()
            );
        }
    }
}
```

```java
package com.acme.billing;

import com.acme.common.api.ApiResponse;
import jakarta.validation.Valid;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/invoices")
    public ResponseEntity<ApiResponse<InvoiceDtos.InvoiceResponse>> createInvoice(
            @Valid @RequestBody InvoiceDtos.CreateInvoiceRequest request
    ) {
        Invoice invoice = invoiceService.create(
                request.customerId(),
                request.amountCents(),
                request.note()
        );
        return ResponseEntity
                .status(HttpStatus.CREATED)
                .body(ApiResponse.ok(InvoiceDtos.InvoiceResponse.from(invoice)));
    }
}
```

Convention decisions: used constructor injection (no field injection/Lombok) and a `record`-based `InvoiceDtos` holder with `jakarta.validation` annotations per the skill; wrapped the response in `ApiResponse.ok(...)` with `201 Created` since this is a resource-creation endpoint rather than a list, so pagination doesn't apply.
withrun 31.00 5 turns · $0.10 · 36 s · claude-sonnet-5
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment✓ skill-fired✓ valid-annotation
Judge reasons
  • reviewer-judgment, DTOs are properly defined as Java records in a BillingDtos holder class, request validation uses @Valid and all required jakarta.validation constraint annotations (@Positive, @Min(1), @Size(max=200)) matching the stated rules, and response is wrapped in ApiResponse.ok().
Tool calls (3)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Bash {"command":"find / -iname \"ApiResponse*.java\" -not -path \"*/node_modules/*\" 2>/dev/null | head -20","description":"Search filesystem for existing ApiResponse class"}
  3. Write {"file_path":"/tmp/eval-shim-ws-spring-request-dtos-are-validated-records-J2E3mL/BillingDtos.java","content":"package com.acme.billing;\n\nimport jakarta.validation.constraints.NotNull;\nimport jakarta.validation.constra
Response
Since this needs full Java source printed in the reply (not committed to files), here's the implementation following the repo's Spring Boot conventions — DTOs as records in a `BillingDtos` holder, constructor injection, `@Valid` validation, and responses wrapped in `ApiResponse`.

```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

public class BillingDtos {

    private BillingDtos() {
    }

    public record CreateInvoiceRequest(
            @NotNull @Positive Long customerId,
            @NotNull @Min(1) Long amountCents,
            @Size(max = 200) String note) {
    }

    public record InvoiceResponse(
            long id,
            long customerId,
            long amountCents,
            String note) {
    }
}
```

```java
package com.acme.billing;

import jakarta.validation.Valid;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;

import com.acme.common.ApiResponse;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/invoices")
    @ResponseStatus(HttpStatus.CREATED)
    public ApiResponse<BillingDtos.InvoiceResponse> createInvoice(
            @Valid @RequestBody BillingDtos.CreateInvoiceRequest request) {
        Invoice invoice = invoiceService.create(
                request.customerId(), request.amountCents(), request.note());

        BillingDtos.InvoiceResponse body = new BillingDtos.InvoiceResponse(
                invoice.id(), invoice.customerId(), invoice.amountCents(), invoice.note());

        return ApiResponse.ok(body);
    }
}
```

Two assumptions I had to make since they're outside the given scope: `InvoiceService.create(...)` returns some type I've called `Invoice` exposing `id()`, `customerId()`, `amountCents()`, `note()` accessors — swap the mapping in `createInvoice` if the real return type differs; and `com.acme.common.ApiResponse<T>.ok(T)` already exists per the house convention, returning a `201 Created` body via `@ResponseStatus`.

Convention decisions applied: constructor injection (no Lombok/field injection), `jakarta.validation` annotations on the request record (`@Positive` for `customerId`, `@Min(1)` for `amountCents` matching "at least 1" exactly, `@Size(max = 200)` for `note`), and the response wrapped in `ApiResponse` rather than returned raw.

spring-service-owns-rules-and-errors

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 0.93 · noise ±0.20

skillspringskill/spring-boot-conventions/business-rules-and-authorization-checks-liveskill/spring-boot-conventions/services-return-records-or-small-resultskill/spring-boot-conventions/throw-domain-exceptions-a-restcontrolleradvice-handler

What this case evaluates, Proves business rules and authorization live in the service (not the controller), failures are domain exceptions mapped by a @RestControllerAdvice handler, and services return records rather than Map<String, Object>; exercises the spring-boot-conventions skill.
The request given to the agent
Add invoice cancellation to our Spring Boot billing module, package `com.acme.billing`.
The rule: a customer may cancel only their own invoice, and only while it is still unpaid.
Assume an `InvoiceRepository` with `Optional<Invoice> findById(long id)` exists and that the
caller passes the current customer's id as `long customerId`; `Invoice` has `customerId()`,
`status()` and `markCancelled()`. Write the service class that implements this, and whatever
else is needed for a failed cancellation to reach the API client as a proper error response.
Print the complete Java source in your reply.
The checks (5)
advice-handlerregexA @RestControllerAdvice handler maps domain exceptions to the error response.
must contain @RestControllerAdvice in last_message
no-map-resultregexThe service does not smuggle results or errors through Map&lt;String, Object&gt;.
must NOT contain Map<String,\s*Object> in last_message
reviewer-judgmentllmJudge the service against the "two locks on the door" and error-handling house conventions.
judge model: The ownership check (customerId matches the invoice's customer) and the unpaid-status rule are both enforced inside the service method itself — not left to the controller or to security configuration — and each violation throws a distinct domain exception. Errors reach the client through a @RestControllerAdvice handler producing ApiResponse.error(...), with no try/catch in any controller. The service never returns Map<String, Object> (void, a record, or a small result type are all fine). Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
throws-domain-exceptionregexRule violations are raised as thrown domain exceptions, not encoded in the return value.
must contain throw\s+new\s+[A-Z]\w*Exception\( in last_message
withrun 11.00 4 turns · $0.12 · 53 s · claude-sonnet-5
✓ advice-handler✓ no-map-result✓ reviewer-judgment✓ skill-fired✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, All five criteria met: ownership and unpaid-status checks enforced in service (not controller/security layer), each violation throws a distinct domain exception, @RestControllerAdvice handler produces ApiResponse.error(...), pattern design eliminates need for controller try/catch, and service returns void rather than Map<String, Object>.
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Bash {"command":"true","description":"noop"}
Response
Here's the complete implementation:

```java
package com.acme.billing;

public class InvoiceNotFoundException extends RuntimeException {

    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice " + invoiceId + " was not found");
    }
}
```

```java
package com.acme.billing;

public class InvoiceAccessDeniedException extends RuntimeException {

    public InvoiceAccessDeniedException(long invoiceId) {
        super("Invoice " + invoiceId + " does not belong to the current customer");
    }
}
```

```java
package com.acme.billing;

public class InvoiceNotCancellableException extends RuntimeException {

    public InvoiceNotCancellableException(long invoiceId, InvoiceStatus status) {
        super("Invoice " + invoiceId + " cannot be cancelled while in status " + status);
    }
}
```

```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

@RestControllerAdvice
public class BillingExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotFound(InvoiceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND).body(ApiResponse.error(ex.getMessage()));
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ResponseEntity<ApiResponse<Void>> handleAccessDenied(InvoiceAccessDeniedException ex) {
        return ResponseEntity.status(HttpStatus.FORBIDDEN).body(ApiResponse.error(ex.getMessage()));
    }

    @ExceptionHandler(InvoiceNotCancellableException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotCancellable(InvoiceNotCancellableException ex) {
        return ResponseEntity.status(HttpStatus.CONFLICT).body(ApiResponse.error(ex.getMessage()));
    }
}
```

```java
package com.acme.billing;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class InvoiceCancellationService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceCancellationService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    @Transactional
    public void cancelInvoice(long invoiceId, long customerId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotCancellableException(invoiceId, invoice.status());
        }

        invoice.markCancelled();
    }
}
```

Notes on assumptions: `invoice.status()` is checked against an existing `InvoiceStatus.UNPAID` enum constant, and `ApiResponse`/`ApiResponse.error(...)` are the existing shared response wrapper/convention already in use elsewhere in the codebase — neither is redefined here, same treatment as the given `Invoice`/`InvoiceRepository`. Authorization is checked in the service (not just relying on the controller/security layer) per the "two locks on the door" convention, with ownership, existence, and state mapped to distinct exceptions (403/404/409) so the controller needs no try/catch.
withrun 21.00 4 turns · $0.13 · 52 s · claude-sonnet-5
✓ advice-handler✓ no-map-result✓ reviewer-judgment✓ skill-fired✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, All criteria met: ownership and unpaid-status checks are both in the service throwing distinct exceptions (InvoiceAccessDeniedException, InvoiceNotCancellableException), errors flow through @RestControllerAdvice to ApiResponse.error(...), no try/catch in the controller, and the service returns void (not Map<String, Object>).
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. ToolSearch {"query":"select:SendMessage","max_results":1}
Response
Here's the complete implementation.

```java
package com.acme.billing;

public final class InvoiceNotFoundException extends RuntimeException {
    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice " + invoiceId + " not found");
    }
}
```

```java
package com.acme.billing;

public final class InvoiceAccessDeniedException extends RuntimeException {
    public InvoiceAccessDeniedException(long invoiceId, long customerId) {
        super("Customer " + customerId + " may not access invoice " + invoiceId);
    }
}
```

```java
package com.acme.billing;

public final class InvoiceNotCancellableException extends RuntimeException {
    public InvoiceNotCancellableException(long invoiceId) {
        super("Invoice " + invoiceId + " is not unpaid and cannot be cancelled");
    }
}
```

```java
package com.acme.billing;

public record ApiResponse<T>(boolean success, T data, ApiError error) {

    public static <T> ApiResponse<T> ok(T data) {
        return new ApiResponse<>(true, data, null);
    }

    public static ApiResponse<Void> error(String code, String message) {
        return new ApiResponse<>(false, null, new ApiError(code, message));
    }

    public record ApiError(String code, String message) {
    }
}
```

```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

@RestControllerAdvice
public class BillingExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotFound(InvoiceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND)
                .body(ApiResponse.error("INVOICE_NOT_FOUND", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ResponseEntity<ApiResponse<Void>> handleAccessDenied(InvoiceAccessDeniedException ex) {
        return ResponseEntity.status(HttpStatus.FORBIDDEN)
                .body(ApiResponse.error("INVOICE_ACCESS_DENIED", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceNotCancellableException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotCancellable(InvoiceNotCancellableException ex) {
        return ResponseEntity.status(HttpStatus.CONFLICT)
                .body(ApiResponse.error("INVOICE_NOT_CANCELLABLE", ex.getMessage()));
    }
}
```

```java
package com.acme.billing;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class InvoiceCancellationService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceCancellationService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    @Transactional
    public void cancel(long invoiceId, long customerId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId, customerId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotCancellableException(invoiceId);
        }

        invoice.markCancelled();
    }
}
```

```java
package com.acme.billing;

import org.springframework.security.core.Authentication;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/invoices")
public class InvoiceController {

    private final InvoiceCancellationService invoiceCancellationService;

    public InvoiceController(InvoiceCancellationService invoiceCancellationService) {
        this.invoiceCancellationService = invoiceCancellationService;
    }

    @PostMapping("/{invoiceId}/cancel")
    public ApiResponse<Void> cancel(@PathVariable long invoiceId, Authentication authentication) {
        long customerId = Long.parseLong(authentication.getName());
        invoiceCancellationService.cancel(invoiceId, customerId);
        return ApiResponse.ok(null);
    }
}
```

Assumptions beyond what was given: `InvoiceStatus` is an existing enum with an `UNPAID` constant (ownership/status fields follow your existing types), and the controller resolves `customerId` from `Authentication.getName()` — swap that line for whatever principal-extraction mechanism your auth stack actually uses.

Convention decisions applied: constructor injection with no Lombok/field injection, and domain exceptions + `@RestControllerAdvice` + `ApiResponse` so the controller stays thin with no try/catch, per the house rules.
withrun 31.00 4 turns · $0.15 · 70 s · claude-sonnet-5
✓ advice-handler✓ no-map-result✓ reviewer-judgment✓ skill-fired✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, Service enforces both ownership and unpaid-status checks with distinct domain exceptions; GlobalExceptionHandler converts them to ApiResponse.error(...) via @RestControllerAdvice with no try/catch in the controller; service returns void.
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Bash {"command":"find /tmp/eval-shim-ws-spring-service-owns-rules-and-errors-fCqzsn -maxdepth 4 2>/dev/null | head -100","description":"Check existing project structure"}
Response
Empty directory — no existing project to match against, so I'll design the full vertical slice (exceptions → advice → service) from the stated conventions and print the source directly, as requested, rather than writing files.

**Design:** three domain exceptions (not found / wrong owner / wrong state), a `@RestControllerAdvice` mapping them to HTTP status + `ApiResponse.error`, and the service doing both authorization checks itself ("two locks on the door") even though a controller/security layer would also gate the route. A thin controller is included to show the full path to the client.

```java
package com.acme.billing;

public enum InvoiceStatus {
    UNPAID,
    PAID,
    CANCELLED
}
```

```java
package com.acme.billing;

import java.util.Optional;

public interface InvoiceRepository {
    Optional<Invoice> findById(long id);
}
```

```java
package com.acme.billing;

public interface Invoice {
    long customerId();
    InvoiceStatus status();
    void markCancelled();
}
```

```java
package com.acme.billing.common;

public record ApiResponse<T>(boolean success, T data, ApiError error) {

    public static <T> ApiResponse<T> ok(T data) {
        return new ApiResponse<>(true, data, null);
    }

    public static <T> ApiResponse<T> error(String code, String message) {
        return new ApiResponse<>(false, null, new ApiError(code, message));
    }

    public record ApiError(String code, String message) {}
}
```

```java
package com.acme.billing.exception;

public class InvoiceNotFoundException extends RuntimeException {
    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice " + invoiceId + " not found");
    }
}
```

```java
package com.acme.billing.exception;

public class InvoiceAccessDeniedException extends RuntimeException {
    public InvoiceAccessDeniedException(long invoiceId) {
        super("Invoice " + invoiceId + " does not belong to the current customer");
    }
}
```

```java
package com.acme.billing.exception;

import com.acme.billing.InvoiceStatus;

public class InvoiceNotCancellableException extends RuntimeException {
    public InvoiceNotCancellableException(long invoiceId, InvoiceStatus status) {
        super("Invoice " + invoiceId + " cannot be cancelled because its status is " + status);
    }
}
```

```java
package com.acme.billing.exception;

import com.acme.billing.common.ApiResponse;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

@RestControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotFound(InvoiceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND)
                .body(ApiResponse.error("INVOICE_NOT_FOUND", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ResponseEntity<ApiResponse<Void>> handleAccessDenied(InvoiceAccessDeniedException ex) {
        return ResponseEntity.status(HttpStatus.FORBIDDEN)
                .body(ApiResponse.error("INVOICE_ACCESS_DENIED", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceNotCancellableException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotCancellable(InvoiceNotCancellableException ex) {
        return ResponseEntity.status(HttpStatus.CONFLICT)
                .body(ApiResponse.error("INVOICE_NOT_CANCELLABLE", ex.getMessage()));
    }
}
```

```java
package com.acme.billing;

import com.acme.billing.exception.InvoiceAccessDeniedException;
import com.acme.billing.exception.InvoiceNotCancellableException;
import com.acme.billing.exception.InvoiceNotFoundException;
import org.springframework.stereotype.Service;

@Service
public class InvoiceService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    public void cancelInvoice(long customerId, long invoiceId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotCancellableException(invoiceId, invoice.status());
        }

        invoice.markCancelled();
    }
}
```

```java
package com.acme.billing;

import com.acme.billing.common.ApiResponse;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/{invoiceId}/cancel")
    public ResponseEntity<ApiResponse<Void>> cancel(
            @PathVariable long invoiceId,
            @AuthenticationPrincipal CustomerPrincipal principal) {
        invoiceService.cancelInvoice(principal.customerId(), invoiceId);
        return ResponseEntity.ok(ApiResponse.ok(null));
    }
}
```

```java
package com.acme.billing;

public record CustomerPrincipal(long customerId) {}
```

Notes on the two non-obvious calls: `InvoiceController` assumes your security layer resolves an authenticated `CustomerPrincipal` (swap for whatever your `UserDetails`/JWT principal actually is) — everything else follows the given method signatures exactly. The service checks ownership and status itself rather than trusting the controller/security layer, per the "two locks on the door" convention, so the rule holds even if called from elsewhere.

spring-work-triggers-skill

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 1.00 · noise ±0.00

skillspringtrigger

What this case evaluates, The tripwire for \"the skill stopped working\": on plainly in-scope Spring work, the spring-boot-conventions skill must fire. If plugin loading breaks (a Claude Code update, a renamed skill, a drifted trigger description), this is the case that goes red first — cleanly, before the content cases fail in confusing ways. Twin of the vue negative-trigger case.
The request given to the agent
Quick one: sketch a Spring Boot endpoint `GET /api/billing/invoices/{id}` for our billing module,
package `com.acme.billing`, assuming `InvoiceService.getInvoice(long id)` exists. Print the Java
source in your reply.
The checks (1)
skill-firedtool_usedThe spring-boot-conventions skill fired on plainly in-scope Spring work. If this fails while the content cases still pass, the skill's *trigger* broke (loading, name, or description) — not its content.
tool Skill matching spring-boot-conventions used 1+ times
withrun 11.00 4 turns · $0.08 · 21 s · claude-sonnet-5
✓ skill-fired
Tool calls (2)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
  2. Write {"file_path":"/tmp/eval-shim-ws-spring-work-triggers-skill-kJ9BPP/InvoiceController.java","content":"package com.acme.billing;\n\nimport org.springframework.web.bind.annotation.GetMapping;\nimport org.springframework.web
Response
Here's the sketch, following the house conventions (constructor injection, wrapped `ApiResponse`, thin controller, no try/catch):

```java
package com.acme.billing;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices/{id}")
    public ApiResponse<BillingDtos.InvoiceResponse> getInvoice(@PathVariable long id) {
        BillingDtos.InvoiceResponse invoice = invoiceService.getInvoice(id);
        return ApiResponse.ok(invoice);
    }
}
```

```java
package com.acme.billing;

public final class BillingDtos {

    private BillingDtos() {}

    public record InvoiceResponse(
        long id,
        String status,
        long amountCents,
        String currency
    ) {}
}
```

Notes:
- Assumed `InvoiceService.getInvoice(long id)` already returns `BillingDtos.InvoiceResponse` (or an equivalent record) rather than an entity/`Map`, per the "services return records, never `Map`" rule; adjust the record fields to match your real invoice shape.
- No try/catch here — a not-found case should throw a domain exception (e.g. `InvoiceNotFoundException`) caught by your existing `@RestControllerAdvice`, which isn't shown since it's presumably already in place elsewhere in the codebase.
withrun 21.00 3 turns · $0.06 · 11 s · claude-sonnet-5
✓ skill-fired
Tool calls (1)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
Response
```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.acme.common.ApiResponse;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices/{id}")
    public ResponseEntity<ApiResponse<?>> getInvoice(@PathVariable long id) {
        var invoice = invoiceService.getInvoice(id);
        return ResponseEntity.ok(ApiResponse.ok(invoice));
    }
}
```

Convention notes: used constructor injection (no Lombok/`@Autowired`), kept the controller thin — no try/catch, since any not-found/domain exception from `InvoiceService.getInvoice` is expected to be mapped by the existing `@RestControllerAdvice`; the single result is wrapped in `ApiResponse.ok(...)` per house rules rather than returned raw.
withrun 31.00 3 turns · $0.06 · 19 s · claude-sonnet-5
✓ skill-fired
Tool calls (1)
  1. Skill {"skill":"komo-stack:spring-boot-conventions"}
Response
```java
package com.acme.billing;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class BillingController {

    private final InvoiceService invoiceService;

    public BillingController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices/{id}")
    public ApiResponse<BillingDtos.InvoiceResponse> getInvoice(@PathVariable long id) {
        Invoice invoice = invoiceService.getInvoice(id);
        return ApiResponse.ok(BillingDtos.InvoiceResponse.from(invoice));
    }
}
```

```java
package com.acme.billing;

import java.math.BigDecimal;
import java.time.LocalDate;

public class BillingDtos {

    public record InvoiceResponse(
            long id,
            String status,
            BigDecimal amount,
            LocalDate dueDate
    ) {
        public static InvoiceResponse from(Invoice invoice) {
            return new InvoiceResponse(
                    invoice.getId(),
                    invoice.getStatus(),
                    invoice.getAmount(),
                    invoice.getDueDate()
            );
        }
    }
}
```

Notes:
- Constructor injection for `InvoiceService`, response wrapped in `ApiResponse.ok(...)`, no raw entity returned — the controller maps to an `InvoiceResponse` record in a `BillingDtos` holder per house convention.
- "Not found" handling is assumed to live in `InvoiceService` (throwing a domain exception caught by a `@RestControllerAdvice`), so the controller stays free of try/catch. `Invoice` field names (`status`, `amount`, `dueDate`) are illustrative — adjust `InvoiceResponse.from` to match your real domain type.

vue-request-does-not-trigger-skill

run 1: 1.00run 2: 1.00run 3: 1.00stable · 1.00 from 1.00 · noise ±0.00

skillnegative-triggervue

What this case evaluates, Proves the Spring skill stays out of frontend requests; exercises the skill's trigger description (negative case).
The request given to the agent
Write a Pinia store (Vue 3, Composition API, TypeScript) for a shopping cart with add, remove, and a total getter. Print the complete file.
The checks (2)
is-pinia-storeregexOutput is a Pinia store.
must contain defineStore\( in last_message
skill-not-firedtool_usedThe Spring skill must NOT activate on a frontend request.
tool Skill matching spring-boot-conventions used 0 times (both arm)
withrun 11.00 2 turns · $0.06 · 8.8 s · claude-sonnet-5
✓ is-pinia-store✓ skill-not-fired
Tool calls (1)
  1. Write {"file_path":"/tmp/eval-shim-ws-vue-request-does-not-trigger-skill-rFMMyD/stores/cart.ts","content":"import { defineStore } from 'pinia'\nimport { computed, ref } from 'vue'\n\nexport interface CartItem {\n id: string\n
Response
Here's the complete file:

```typescript
// stores/cart.ts
import { defineStore } from 'pinia'
import { computed, ref } from 'vue'

export interface CartItem {
  id: string
  name: string
  price: number
  quantity: number
}

export const useCartStore = defineStore('cart', () => {
  const items = ref<CartItem[]>([])

  function addItem(item: Omit<CartItem, 'quantity'>, quantity = 1) {
    const existing = items.value.find((i) => i.id === item.id)
    if (existing) {
      existing.quantity += quantity
    } else {
      items.value.push({ ...item, quantity })
    }
  }

  function removeItem(id: string) {
    items.value = items.value.filter((i) => i.id !== id)
  }

  const total = computed(() =>
    items.value.reduce((sum, i) => sum + i.price * i.quantity, 0)
  )

  return { items, addItem, removeItem, total }
})
```

Note: I wasn't able to write this to disk since file writes are blocked in this session's permission mode — paste it in as `stores/cart.ts` yourself.
withrun 21.00 1 turns · $0.05 · 7.8 s · claude-sonnet-5
✓ is-pinia-store✓ skill-not-fired
Tool calls (0)

none

Response
```ts
// stores/cart.ts
import { ref, computed } from 'vue'
import { defineStore } from 'pinia'

export interface CartItem {
  id: string
  name: string
  price: number
  quantity: number
}

export const useCartStore = defineStore('cart', () => {
  const items = ref<CartItem[]>([])

  function addItem(item: Omit<CartItem, 'quantity'>, quantity = 1) {
    const existing = items.value.find((i) => i.id === item.id)
    if (existing) {
      existing.quantity += quantity
    } else {
      items.value.push({ ...item, quantity })
    }
  }

  function removeItem(id: string, quantity?: number) {
    const existing = items.value.find((i) => i.id === id)
    if (!existing) return

    if (quantity === undefined || existing.quantity <= quantity) {
      items.value = items.value.filter((i) => i.id !== id)
    } else {
      existing.quantity -= quantity
    }
  }

  function clear() {
    items.value = []
  }

  const total = computed(() =>
    items.value.reduce((sum, item) => sum + item.price * item.quantity, 0)
  )

  const itemCount = computed(() =>
    items.value.reduce((count, item) => count + item.quantity, 0)
  )

  return {
    items,
    addItem,
    removeItem,
    clear,
    total,
    itemCount,
  }
})
```

Notes: `addItem` merges into an existing line by quantity; `removeItem` takes an optional `quantity` (removes that many, or the whole line if omitted/exceeded). `total` and `itemCount` are reactive getters derived via `computed`.
withrun 31.00 1 turns · $0.04 · 6.5 s · claude-sonnet-5
✓ is-pinia-store✓ skill-not-fired
Tool calls (0)

none

Response
```ts
// stores/cart.ts
import { defineStore } from 'pinia'
import { computed, ref } from 'vue'

export interface CartItem {
  id: string
  name: string
  price: number
  quantity: number
}

export const useCartStore = defineStore('cart', () => {
  const items = ref<CartItem[]>([])

  function addItem(item: Omit<CartItem, 'quantity'>, quantity = 1) {
    const existing = items.value.find((i) => i.id === item.id)
    if (existing) {
      existing.quantity += quantity
    } else {
      items.value.push({ ...item, quantity })
    }
  }

  function removeItem(id: string) {
    items.value = items.value.filter((i) => i.id !== id)
  }

  const total = computed(() =>
    items.value.reduce((sum, i) => sum + i.price * i.quantity, 0)
  )

  return {
    items,
    addItem,
    removeItem,
    total,
  }
})
```

Notes: `addItem` merges into an existing line item by `id` (bumping `quantity`) rather than creating duplicates; `removeItem` drops the whole line item by `id`. Swap `id: string` for `number` if that matches your product IDs.

Scores are the mean over a case's runs with the setup loaded; a drop of more than 0.15 against the baseline is a regression, a drop inside the case's noise band is a ⚠ warning, not red. Indicators (ind) are recorded but not scored. Generated by config-drift-checker.