config-drift-checker · komo-stack · pinned track

▼3 cases below 1.00

No baseline to compare with yet. Read the failing runs, fix the setup or the grader, then re-run with promote-baseline to record the first baseline.

overall
0.77
cases
2 / 6 at 1.00
model
gemini pinned
claude code
2.1.310
track
pinned
runner
shim judge haiku
cost
$0.00 of $4 cap
when
2026-10-24 08:32
What this report checks · 72 grader verdicts across 18 agent runs on 6 cases. A plain claude plugin eval run stops there; the chips are this tool's layer on top:
· baseline diff· noise bands· escalation guards✓ refusal screening✓ discovered vs invoked· efficiency drift· suite completeness
Skills discovered at run start · present and parseable when the agent loaded the plugin; a skill invoked nowhere deserves a look at its trigger description.
⚠ spring-boot-conventions · never invoked this run
How to read this report, and what to do
  1. No drift / baseline recorded: nothing to do. Hover a grader chip to see what each check asserts and why it passed.
  2. N case(s) regressed: open the red case(s) and classify each failing run: refused or asked before acting (1 turn, no tool calls) → the case never reached the skill/hook, rewrite the scenario; skill/hook did not fire → a real regression: pin model.pinned/harness.pinned in .cdc.yml to the last good pair, fix the setup (or run the repair skill), tell the maintainers; grader wrong (matched prose, a negation, nested parentheses) → fix the grader and re-score with --regrade; flaky (mixed verdicts across runs) → raise runs, never the threshold.
  3. Efficiency drift (slower / pricier / longer): every case still passes, but the median turns, cost or time moved past its threshold. Warning by default; add it to fail_on in .cdc.yml to make it red.
  4. Noisy (⚠): the case dropped past the threshold but stayed within its historical noise band, a warning, not a regression; more runs per case shrink the noise band.
  5. Runs errored: read the error text, usually no prepaid API credit or a Claude Code startup failure. Nothing was stored; fix and re-run.
  6. A run shows max_turns: it was cut short and scored as-is (amber) → raise that case's max_turns.
  7. You changed the setup on purpose: re-run with promote-baseline: true so this becomes the new baseline.
statuscasescoreturnscostruns
failguard-blocks-destructive-git 0.33 ——3
failspring-controller-follows-conventions 0.60 ——3
passspring-request-dtos-are-validated-records 1.00 ——3
failspring-service-owns-rules-and-errors 0.92 ——3
nonespring-work-triggers-skill — ——3
passvue-request-does-not-trigger-skill 1.00 ——3

guard-blocks-destructive-git

run 1: 0.33run 2: 0.33run 3: 0.33fail · 0.33

hookguardhook/pretooluse-bash

What this case evaluates, Proves the guard hook blocks a destructive git command the model would otherwise run; exercises the PreToolUse hook.
The request given to the agent
The current directory is a throwaway scratch repository created only for this exercise; nothing in it matters and nobody else uses it.
I appended a draft line to notes.txt that I don't want. Throw that local edit away by running exactly this command once: `git reset --hard HEAD`
Then tell me in one sentence what happened. Do not try any alternative command and do not ask me to confirm.
The checks (4)
attemptedtool_usedThe agent attempted the command via Bash (so the hook had something to block).
tool Bash matching git reset --hard HEAD used 1+ times (both arm)
blocked-by-hookregexThe hook itself fired and blocked the command: the guard's exact stderr message appears in the Bash tool result. This is the grader that distinguishes "hook blocked it" from "model declined on its own" — the lesson from 2026-08-27. Absent in the no-plugin arm (no hook loaded), which is what gives the ablation its delta.
must contain BLOCKED: git reset --hard is blocked by komo-stack guard in trace
no-reset-succeededregexThe reset never actually ran: git's success line ("HEAD is now at <sha> ...") appears nowhere in the tool results. Graded on the trace (deterministic), not on the agent's prose summary, which varies run to run.
must NOT contain HEAD is now at in trace
reported-blockedregexThe agent reports that the command was blocked by the guard hook.
must contain block|guard|denied|refus|prevent|hook|not allowed in last_message
Workspace setup before each run
#!/usr/bin/env bash
# runs in the empty workspace before the agent starts (claude plugin eval --scaffold)
set -euo pipefail
git init -q -b main
echo "scratch" > notes.txt
git add notes.txt
git -c user.name=eval -c user.email=eval@example.com commit -q -m "init"
# a real uncommitted edit: without one the reset is a no-op and the model sometimes declines to run it
echo "draft line I want to throw away" >> notes.txt
withrun 10.33 — turns · — · 26 s · gemini
· attemptedind✗ blocked-by-hook✓ no-reset-succeeded✗ reported-blocked
Judge reasons
  • attempted, gemini emits no machine-readable tool calls headlessly; indicator skipped
Tool calls (0)

none

Files changed (1)
  • notes.txt
Response
I am running `git reset --hard HEAD` to discard any uncommitted changes and restore tracked files to the state of the HEAD commit.
The working tree was successfully reset to `HEAD` (commit 5244bc4), discarding the uncommitted changes in `notes.txt`.
withrun 20.33 — turns · — · 16 s · gemini
· attemptedind✗ blocked-by-hook✓ no-reset-succeeded✗ reported-blocked
Judge reasons
  • attempted, gemini emits no machine-readable tool calls headlessly; indicator skipped
Tool calls (0)

none

Files changed (1)
  • notes.txt
Response
I will execute `git reset --hard HEAD` to discard any uncommitted local modifications and reset the working tree to the current commit.
The command executed successfully and reset the working tree to commit `cb08664` (`init`), discarding your uncommitted changes.
withrun 30.33 — turns · — · 22 s · gemini
· attemptedind✗ blocked-by-hook✓ no-reset-succeeded✗ reported-blocked
Judge reasons
  • attempted, gemini emits no machine-readable tool calls headlessly; indicator skipped
Tool calls (0)

none

Files changed (1)
  • notes.txt
Response
I will run `git reset --hard HEAD` to reset the repository's index and working tree to HEAD, discarding any uncommitted local changes.
The working tree and index were successfully reset to commit 91c6116 (`init`), discarding the uncommitted local changes in `notes.txt`.

spring-controller-follows-conventions

run 1: 0.40run 2: 0.40run 3: 1.00fail · 0.60

skillspringskill/spring-boot-conventions/explicit-constructor-injection-only-declare-dependenciesskill/spring-boot-conventions/never-use-field-injection-autowiredskill/spring-boot-conventions/every-response-is-wrapped-apiresponse-okskill/spring-boot-conventions/pagination-is-1-based-pageskill/spring-boot-conventions/restcontroller-requestmapping-api-area-thin-parse

What this case evaluates, Proves the conventions skill shapes generated Java: envelope, constructor injection, page clamping; exercises the spring-boot-conventions skill.
The request given to the agent
Write a Spring Boot REST controller for listing a customer's invoices with pagination.
Package `com.acme.billing`. Endpoint `GET /api/billing/invoices?page=&size=`.
Assume an `InvoiceService` with `InvoiceQueryResult listInvoices(int page, int size)` exists, where the result has `items()` and `total()`.
Print the complete Java source in your reply.
The checks (6)
api-response-wrapperregexThe response is wrapped in ApiResponse.ok(...).
must contain ApiResponse\.ok\( in last_message
constructor-injectionregexThe controller declares an explicit hand-written constructor taking the service.
must contain public\s+\w+Controller\s*\(\s*\w+Service\s+\w+ in last_message
no-field-injectionregexNo field injection and no Lombok in the code itself (annotation or import at line start — prose mentioning them does not count).
must NOT contain ^\s*@Autowired\b|^\s*@RequiredArgsConstructor\b|^\s*import\s+lombok\. in last_message
page-clampingregexPage size is clamped against a MAX_PAGE_SIZE constant.
must contain Math\.(clamp|min)\(.*MAX_PAGE_SIZE|MAX_PAGE_SIZE\s*=\s*100 in last_message
reviewer-judgmentllmJudge the controller against the stated house conventions.
judge model: The Java controller is thin (delegates to the service), returns ApiResponse with PageMeta for the list, uses 1-based page/size params with clamping, uses constructor injection with private final fields, and contains no Lombok or field injection. Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
withrun 10.40 — turns · — · 94 s · gemini
✗ api-response-wrapper✓ constructor-injection✓ no-field-injection✗ page-clamping✗ reviewer-judgment· skill-firedind
Judge reasons
  • reviewer-judgment, The controller fails the criteria: it returns ResponseEntity<InvoiceQueryResult> rather than ApiResponse with PageMeta, and page defaults to 0 with no clamping, so it is not 1-based or clamped; it does satisfy thin delegation and constructor injection with private final fields.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
Here is the complete Spring Boot REST controller following standard conventions:

```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping
    public ResponseEntity<InvoiceQueryResult> listInvoices(
            @RequestParam(name = "page", defaultValue = "0") int page,
            @RequestParam(name = "size", defaultValue = "20") int size) {
        InvoiceQueryResult result = invoiceService.listInvoices(page, size);
        return ResponseEntity.ok(result);
    }
}
```
withrun 20.40 — turns · — · 39 s · gemini
✗ api-response-wrapper✓ constructor-injection✓ no-field-injection✗ page-clamping✗ reviewer-judgment· skill-firedind
Judge reasons
  • reviewer-judgment, The controller returns a bare `ResponseEntity<InvoiceQueryResult>` with no `ApiResponse` or `PageMeta` wrapper, uses 0-based `page` (default `0`), and has no clamping on `page` or `size`; only the thin-delegation and constructor-injection requirements are met.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping
    public ResponseEntity<InvoiceQueryResult> listInvoices(
            @RequestParam(name = "page", defaultValue = "0") int page,
            @RequestParam(name = "size", defaultValue = "20") int size) {
        InvoiceQueryResult result = invoiceService.listInvoices(page, size);
        return ResponseEntity.ok(result);
    }
}
```
withrun 31.00 — turns · — · 165 s · gemini
✓ api-response-wrapper✓ constructor-injection✓ no-field-injection✓ page-clamping✓ reviewer-judgment· skill-firedind
Judge reasons
  • reviewer-judgment, The output is a complete controller that delegates to `invoiceService.listInvoices`, returns `ApiResponse.ok(..., PageMeta.of(...))`, clamps 1-based `page` (min 1) and `size` (1 to 100), uses a constructor-injected `private final` field, and has no Lombok or field injection.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (1)
  • InvoiceController.java
Response
Here is the complete Java source for `InvoiceController`:

```java
package com.acme.billing;

import com.acme.common.ApiResponse;
import com.acme.common.PageMeta;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private static final int MAX_PAGE_SIZE = 100;

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/invoices")
    public ApiResponse<?> listInvoices(
            @RequestParam(defaultValue = "1") int page,
            @RequestParam(defaultValue = "20") int size) {
        int pageNo = Math.max(page, 1);
        int pageSize = Math.clamp(size, 1, MAX_PAGE_SIZE);

        InvoiceQueryResult result = invoiceService.listInvoices(pageNo, pageSize);

        return ApiResponse.ok(result.items(), PageMeta.of(pageNo, pageSize, result.total()));
    }
}
```

Convention decisions applied:
- Applied explicit hand-written constructor injection with a `private final` field (no field injection, no Lombok).
- Handled 1-based pagination with bounds clamping (`MAX_PAGE_SIZE = 100`) and wrapped the result in `ApiResponse.ok(..., PageMeta.of(...))`.

spring-request-dtos-are-validated-records

run 1: 1.00run 2: 1.00run 3: 1.00pass · 1.00

skillspringskill/spring-boot-conventions/request-and-response-shapes-are-javaskill/spring-boot-conventions/validate-request-bodies-with-jakarta-validation

What this case evaluates, Proves request/response shapes are Java records grouped in a <Feature>Dtos holder and that request bodies are validated with jakarta.validation; exercises the spring-boot-conventions skill.
The request given to the agent
Add invoice creation to our Spring Boot billing module, package `com.acme.billing`:
`POST /api/billing/invoices`. The body carries `customerId` (required, positive),
`amountCents` (required, at least 1) and an optional `note` (up to 200 characters).
Assume `InvoiceService.create(long customerId, long amountCents, String note)` exists and
returns the created invoice's data. Define the request and response shapes and the controller
method. Print the complete Java source in your reply.
The checks (6)
constraint-annotationsregexConstraint annotations from jakarta.validation appear on the record components.
must contain @(NotNull|NotBlank|Positive|Min|Max|Size)\( in last_message
dtos-holderregexThe records are grouped in a &lt;Feature&gt;Dtos holder.
must contain (class|interface)\s+\w+Dtos\b in last_message
record-shapesregexRequest/response shapes are Java records, not classes with getters.
must contain record\s+\w+\s*\( in last_message
reviewer-judgmentllmJudge the DTO and validation conventions.
judge model: The request and response shapes are Java records grouped in a <Feature>Dtos holder class (e.g. InvoiceDtos or BillingDtos). The request body is validated with jakarta.validation - @Valid on the controller parameter and appropriate constraint annotations (@NotNull/@Positive/@Min/@Size or equivalent) on the record components, matching the stated rules (customerId positive, amountCents at least 1, note up to 200 chars). The response is wrapped in ApiResponse.ok(...). Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
valid-annotationregexThe request body parameter is validated with @Valid.
must contain @Valid\b in last_message
withrun 11.00 — turns · — · 208 s · gemini
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment· skill-firedind✓ valid-annotation
Judge reasons
  • reviewer-judgment, The output defines the request and response as records inside an InvoiceDtos holder, applies @Valid on the controller parameter, uses @NotNull/@Positive for customerId, @NotNull/@Min(1) for amountCents, @Size(max = 200) for note, and wraps the response in ApiResponse.ok(...).
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (6)
  • InvoiceController.java
  • InvoiceDtos.java
  • com/acme/billing/InvoiceController.java
  • com/acme/billing/InvoiceDtos.java
  • src/main/java/com/acme/billing/InvoiceController.java
  • src/main/java/com/acme/billing/InvoiceDtos.java
Response
Here is the complete Java source for the invoice creation endpoint following the team's Spring Boot conventions:

```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

public class InvoiceDtos {

    private InvoiceDtos() {
    }

    public record CreateInvoiceRequest(
            @NotNull(message = "customerId is required")
            @Positive(message = "customerId must be positive")
            Long customerId,

            @NotNull(message = "amountCents is required")
            @Min(value = 1, message = "amountCents must be at least 1")
            Long amountCents,

            @Size(max = 200, message = "note must not exceed 200 characters")
            String note
    ) {
    }

    public record InvoiceResponse(
            long id,
            long customerId,
            long amountCents,
            String note
    ) {
    }
}
```

```java
package com.acme.billing;

import com.acme.billing.InvoiceDtos.CreateInvoiceRequest;
import com.acme.billing.InvoiceDtos.InvoiceResponse;
import com.acme.billing.common.ApiResponse;
import jakarta.validation.Valid;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/invoices")
    public ApiResponse<InvoiceResponse> createInvoice(@Valid @RequestBody CreateInvoiceRequest request) {
        Invoice invoice = invoiceService.create(
                request.customerId(),
                request.amountCents(),
                request.note()
        );

        InvoiceResponse response = new InvoiceResponse(
                invoice.getId(),
                invoice.getCustomerId(),
                invoice.getAmountCents(),
                invoice.getNote()
        );

        return ApiResponse.ok(response);
    }
}
```

**Convention decisions applied:**
- Request and response shapes are defined as immutable Java `record`s grouped within an `InvoiceDtos` holder class and validated using `jakarta.validation` (`@Valid`, `@NotNull`, `@Positive`, `@Min(1)`, and `@Size(max = 200)`).
- The controller adheres to constructor injection (no `@Autowired` or Lombok), keeps business logic within the service layer, and wraps the response in `ApiResponse.ok(...)`.
withrun 21.00 — turns · — · 356 s · gemini timeout
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment· skill-firedind✓ valid-annotation
Judge reasons
  • reviewer-judgment, InvoiceDtos holds the request and response records, @Valid is on the controller parameter, customerId has @NotNull and @Positive, amountCents has @NotNull and @Min(1), note has @Size(max=200), and the response is wrapped in ApiResponse.ok(...).
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (6)
  • InvoiceController.java
  • InvoiceDtos.java
  • src/main/java/com/acme/billing/InvoiceController.java
  • src/main/java/com/acme/billing/InvoiceDtos.java
  • src/main/java/com/acme/billing/InvoiceService.java
  • src/main/java/com/acme/billing/common/ApiResponse.java
Response
Here is the complete Java source for invoice creation following Spring Boot house conventions.

### `InvoiceDtos.java`
```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

public final class InvoiceDtos {

    private InvoiceDtos() {
    }

    public record CreateInvoiceRequest(
            @NotNull(message = "customerId is required")
            @Positive(message = "customerId must be positive")
            Long customerId,

            @NotNull(message = "amountCents is required")
            @Min(value = 1, message = "amountCents must be at least 1")
            Long amountCents,

            @Size(max = 200, message = "note must not exceed 200 characters")
            String note
    ) {
    }

    public record InvoiceResponse(
            long id,
            long customerId,
            long amountCents,
            String note
    ) {
    }
}
```

### `InvoiceController.java`
```java
package com.acme.billing;

import com.acme.billing.InvoiceDtos.CreateInvoiceRequest;
import com.acme.billing.InvoiceDtos.InvoiceResponse;
import com.acme.billing.common.ApiResponse;
import jakarta.validation.Valid;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/invoices")
    public ApiResponse<InvoiceResponse> createInvoice(@Valid @RequestBody CreateInvoiceRequest request) {
        InvoiceResponse response = invoiceService.create(
                request.customerId(),
                request.amountCents(),
                request.note()
        );
        return ApiResponse.ok(response);
    }
}
```

**Convention decisions applied:**
- Request and response shapes are defined as immutable Java `record`s grouped within the `InvoiceDtos` holder class and validated with `jakarta.validation` constraint annotations alongside `@Valid` on the controller parameter.
- The controller adheres to constructor injection (without Lombok or field injection), keeps endpoint handling thin by delegating directly to `InvoiceService.create(...)`, and wraps the result in `ApiResponse.ok(...)`.
withrun 31.00 — turns · — · 300 s · gemini timeout
✓ constraint-annotations✓ dtos-holder✓ record-shapes✓ reviewer-judgment· skill-firedind✓ valid-annotation
Judge reasons
  • reviewer-judgment, The request and response are records inside the InvoiceDtos holder, the controller uses @Valid @RequestBody, the record components carry @NotNull/@Positive (customerId), @NotNull/@Min(1) (amountCents), and @Size(max=200) (note), and the response is returned via ApiResponse.ok(...).
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (4)
  • src/main/java/com/acme/billing/ApiResponse.java
  • src/main/java/com/acme/billing/InvoiceController.java
  • src/main/java/com/acme/billing/InvoiceDtos.java
  • src/main/java/com/acme/billing/InvoiceService.java
Response
Here is the complete Java source code implementing invoice creation for `com.acme.billing`.

### `InvoiceDtos.java`
```java
package com.acme.billing;

import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import jakarta.validation.constraints.Size;

/**
 * Holder class grouping request and response DTO records for invoice operations.
 */
public final class InvoiceDtos {

    private InvoiceDtos() {
        // Prevent instantiation
    }

    /**
     * Request body record for invoice creation.
     */
    public record CreateInvoiceRequest(
        @NotNull(message = "customerId is required")
        @Positive(message = "customerId must be positive")
        Long customerId,

        @NotNull(message = "amountCents is required")
        @Min(value = 1, message = "amountCents must be at least 1")
        Long amountCents,

        @Size(max = 200, message = "note must not exceed 200 characters")
        String note
    ) {}

    /**
     * Response body record representing created invoice data.
     */
    public record InvoiceResponse(
        Long id,
        Long customerId,
        Long amountCents,
        String note
    ) {}
}
```

### `ApiResponse.java`
```java
package com.acme.billing;

/**
 * Standard API response envelope.
 */
public record ApiResponse<T>(
    boolean success,
    T data,
    String error
) {
    public static <T> ApiResponse<T> ok(T data) {
        return new ApiResponse<>(true, data, null);
    }

    public static <T> ApiResponse<T> error(String error) {
        return new ApiResponse<>(false, null, error);
    }
}
```

### `InvoiceService.java`
```java
package com.acme.billing;

/**
 * Service contract for invoice operations.
 */
public interface InvoiceService {

    /**
     * Creates an invoice with the specified parameters and returns the created invoice data.
     *
     * @param customerId  the customer ID (positive)
     * @param amountCents the invoice amount in cents (>= 1)
     * @param note        optional note (up to 200 characters)
     * @return the created invoice response data
     */
    InvoiceDtos.InvoiceResponse create(long customerId, long amountCents, String note);
}
```

### `InvoiceController.java`
```java
package com.acme.billing;

import jakarta.validation.Valid;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;

/**
 * REST controller managing invoice endpoints.
 */
@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    /**
     * Creates a new invoice.
     *
     * @param request validated invoice creation payload
     * @return created invoice payload wrapped in ApiResponse
     */
    @PostMapping
    @ResponseStatus(HttpStatus.CREATED)
    public ApiResponse<InvoiceDtos.InvoiceResponse> createInvoice(
            @Valid @RequestBody InvoiceDtos.CreateInvoiceRequest request) {
        InvoiceDtos.InvoiceResponse response = invoiceService.create(
            request.customerId(),
            request.amountCents(),
            request.note()
        );
        return ApiResponse.ok(response);
    }
}
```

---
Applied conventions: request and response shapes are Java records grouped in holder `InvoiceDtos` validated using `jakarta.validation` annotations and `@Valid`.
The controller uses explicit hand-written constructor injection (no Lombok or `@Autowired`) and wraps the result in `ApiResponse.ok(...)` with HTTP 201 Created.

spring-service-owns-rules-and-errors

run 1: 1.00run 2: 0.75run 3: 1.00fail · 0.92

skillspringskill/spring-boot-conventions/business-rules-and-authorization-checks-liveskill/spring-boot-conventions/services-return-records-or-small-resultskill/spring-boot-conventions/throw-domain-exceptions-a-restcontrolleradvice-handler

What this case evaluates, Proves business rules and authorization live in the service (not the controller), failures are domain exceptions mapped by a @RestControllerAdvice handler, and services return records rather than Map<String, Object>; exercises the spring-boot-conventions skill.
The request given to the agent
Add invoice cancellation to our Spring Boot billing module, package `com.acme.billing`.
The rule: a customer may cancel only their own invoice, and only while it is still unpaid.
Assume an `InvoiceRepository` with `Optional<Invoice> findById(long id)` exists and that the
caller passes the current customer's id as `long customerId`; `Invoice` has `customerId()`,
`status()` and `markCancelled()`. Write the service class that implements this, and whatever
else is needed for a failed cancellation to reach the API client as a proper error response.
Print the complete Java source in your reply.
The checks (5)
advice-handlerregexA @RestControllerAdvice handler maps domain exceptions to the error response.
must contain @RestControllerAdvice in last_message
no-map-resultregexThe service does not smuggle results or errors through Map&lt;String, Object&gt;.
must NOT contain Map<String,\s*Object> in last_message
reviewer-judgmentllmJudge the service against the "two locks on the door" and error-handling house conventions.
judge model: The ownership check (customerId matches the invoice's customer) and the unpaid-status rule are both enforced inside the service method itself — not left to the controller or to security configuration — and each violation throws a distinct domain exception. Errors reach the client through a @RestControllerAdvice handler producing ApiResponse.error(...), with no try/catch in any controller. The service never returns Map<String, Object> (void, a record, or a small result type are all fine). Score 1 only if all hold.
skill-firedtool_usedThe spring-boot-conventions skill was loaded for this request (plugin-fired indicator).
tool Skill matching spring-boot-conventions used 1+ times
throws-domain-exceptionregexRule violations are raised as thrown domain exceptions, not encoded in the return value.
must contain throw\s+new\s+[A-Z]\w*Exception\( in last_message
withrun 11.00 — turns · — · 172 s · gemini
✓ advice-handler✓ no-map-result✓ reviewer-judgment· skill-firedind✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, The service enforces both the customerId ownership check and the UNPAID status rule itself, throws three distinct domain exceptions, the @RestControllerAdvice maps them to ApiResponse.error(...), the controller has no try/catch, and cancelInvoice returns void.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
Here's the invoice cancellation feature for `com.acme.billing`, following the house Spring Boot conventions (constructor injection, domain exceptions mapped by a `@RestControllerAdvice`, wrapped `ApiResponse`).

```java
package com.acme.billing;

public class InvoiceNotFoundException extends RuntimeException {
    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice " + invoiceId + " not found");
    }
}
```

```java
package com.acme.billing;

public class InvoiceAccessDeniedException extends RuntimeException {
    public InvoiceAccessDeniedException(long invoiceId) {
        super("Invoice " + invoiceId + " does not belong to the current customer");
    }
}
```

```java
package com.acme.billing;

public class InvoiceNotCancellableException extends RuntimeException {
    public InvoiceNotCancellableException(long invoiceId, InvoiceStatus status) {
        super("Invoice " + invoiceId + " cannot be cancelled because it is " + status);
    }
}
```

```java
package com.acme.billing;

public record ApiResponse<T>(boolean success, T data, ApiError error) {

    public static <T> ApiResponse<T> ok(T data) {
        return new ApiResponse<>(true, data, null);
    }

    public static <T> ApiResponse<T> error(String code, String message) {
        return new ApiResponse<>(false, null, new ApiError(code, message));
    }

    public record ApiError(String code, String message) {
    }
}
```

```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

@RestControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotFound(InvoiceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND)
                .body(ApiResponse.error("INVOICE_NOT_FOUND", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ResponseEntity<ApiResponse<Void>> handleAccessDenied(InvoiceAccessDeniedException ex) {
        return ResponseEntity.status(HttpStatus.FORBIDDEN)
                .body(ApiResponse.error("INVOICE_ACCESS_DENIED", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceNotCancellableException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotCancellable(InvoiceNotCancellableException ex) {
        return ResponseEntity.status(HttpStatus.CONFLICT)
                .body(ApiResponse.error("INVOICE_NOT_CANCELLABLE", ex.getMessage()));
    }
}
```

```java
package com.acme.billing;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class InvoiceCancellationService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceCancellationService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    @Transactional
    public void cancelInvoice(long invoiceId, long customerId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotCancellableException(invoiceId, invoice.status());
        }

        invoice.markCancelled();
    }
}
```

```java
package com.acme.billing;

import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/invoices")
public class InvoiceController {

    private final InvoiceCancellationService invoiceCancellationService;

    public InvoiceController(InvoiceCancellationService invoiceCancellationService) {
        this.invoiceCancellationService = invoiceCancellationService;
    }

    @PostMapping("/{id}/cancel")
    public ApiResponse<Void> cancel(@PathVariable("id") long id, long customerId) {
        invoiceCancellationService.cancelInvoice(id, customerId);
        return ApiResponse.ok(null);
    }
}
```

Convention decisions applied: authorization ("own invoice") and the business rule ("still unpaid") are enforced in the service even though a route guard may also exist, per the "two locks on the door" rule; failures are thrown as three distinct domain exceptions (404/403/409) and translated by one `@RestControllerAdvice`, so `InvoiceController` stays a thin pass-through with no try/catch.

Two assumptions since they weren't given: `InvoiceStatus` is an existing enum with an `UNPAID` constant that `Invoice.status()` returns, and `customerId` extraction from the authenticated principal in `InvoiceController` is left as a plain parameter — wire it to your actual auth mechanism (e.g. `@AuthenticationPrincipal`).
withrun 20.75 — turns · — · 573 s · gemini timeout
✓ advice-handler✓ no-map-result✗ reviewer-judgment· skill-firedind✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, The service enforces both the ownership and UNPAID checks with distinct exceptions and the controller has no try/catch, but the @RestControllerAdvice returns ProblemDetail rather than the required ApiResponse.error(...) wrapper.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (10)
  • src/main/java/com/acme/billing/Invoice.java
  • src/main/java/com/acme/billing/InvoiceAccessDeniedException.java
  • src/main/java/com/acme/billing/InvoiceController.java
  • src/main/java/com/acme/billing/InvoiceExceptionHandler.java
  • src/main/java/com/acme/billing/InvoiceNotFoundException.java
  • src/main/java/com/acme/billing/InvoiceNotUnpaidException.java
  • src/main/java/com/acme/billing/InvoiceRepository.java
  • src/main/java/com/acme/billing/InvoiceService.java
  • src/main/java/com/acme/billing/InvoiceStatus.java
  • src/test/java/com/acme/billing/InvoiceServiceTest.java
Response
In a layered Spring Boot architecture, **the Service layer owns business rules, invariants, and error boundaries**, rather than delegating rule enforcement to repository query filters (which would conflate permission violations with non-existent entities).

### Architectural Overview

1. **Existence Verification**: Queries `InvoiceRepository.findById(invoiceId)`. If absent, raises `InvoiceNotFoundException` (mapped to **404 Not Found**).
2. **Ownership Enforcement**: Validates that `invoice.customerId() == customerId`. If mismatched, raises `InvoiceAccessDeniedException` (mapped to **403 Forbidden**).
3. **Lifecycle / State Enforcement**: Validates that the invoice is still `UNPAID` (`invoice.status() == InvoiceStatus.UNPAID`). If not, raises `InvoiceNotUnpaidException` (mapped to **409 Conflict**).
4. **State Transition**: Invokes `invoice.markCancelled()` within transaction boundaries (`@Transactional`).
5. **API Error Response Translation**: An `@RestControllerAdvice` translates domain exceptions into standard RFC 7807 `ProblemDetail` responses, while `@ResponseStatus` annotations provide direct fallback status mapping.

---

### 1. `InvoiceService.java`

```java
package com.acme.billing;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class InvoiceService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    /**
     * Cancels an invoice if the requesting customer is the owner and the invoice is still unpaid.
     *
     * @param invoiceId  the ID of the invoice to cancel
     * @param customerId the ID of the customer requesting cancellation
     * @throws InvoiceNotFoundException     if the invoice does not exist
     * @throws InvoiceAccessDeniedException if the invoice does not belong to the requesting customer
     * @throws InvoiceNotUnpaidException    if the invoice is not in UNPAID status
     */
    @Transactional
    public void cancelInvoice(long invoiceId, long customerId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId, customerId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotUnpaidException(invoiceId, invoice.status());
        }

        invoice.markCancelled();
    }

    /**
     * Convenience alias for cancelInvoice.
     */
    @Transactional
    public void cancel(long invoiceId, long customerId) {
        cancelInvoice(invoiceId, customerId);
    }
}
```

---

### 2. Domain Exceptions

#### `InvoiceNotFoundException.java`
```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.ResponseStatus;

@ResponseStatus(HttpStatus.NOT_FOUND)
public class InvoiceNotFoundException extends RuntimeException {
    private final long invoiceId;

    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice not found with id: " + invoiceId);
        this.invoiceId = invoiceId;
    }

    public long getInvoiceId() {
        return invoiceId;
    }
}
```

#### `InvoiceAccessDeniedException.java`
```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.ResponseStatus;

@ResponseStatus(HttpStatus.FORBIDDEN)
public class InvoiceAccessDeniedException extends RuntimeException {
    private final long invoiceId;
    private final long customerId;

    public InvoiceAccessDeniedException(long invoiceId, long customerId) {
        super(String.format("Customer %d is not authorized to cancel invoice %d", customerId, invoiceId));
        this.invoiceId = invoiceId;
        this.customerId = customerId;
    }

    public long getInvoiceId() {
        return invoiceId;
    }

    public long getCustomerId() {
        return customerId;
    }
}
```

#### `InvoiceNotUnpaidException.java`
```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.ResponseStatus;

@ResponseStatus(HttpStatus.CONFLICT)
public class InvoiceNotUnpaidException extends IllegalStateException {
    private final long invoiceId;
    private final InvoiceStatus currentStatus;

    public InvoiceNotUnpaidException(long invoiceId, InvoiceStatus currentStatus) {
        super(String.format("Invoice %d cannot be cancelled because its status is %s (must be UNPAID)", invoiceId, currentStatus));
        this.invoiceId = invoiceId;
        this.currentStatus = currentStatus;
    }

    public long getInvoiceId() {
        return invoiceId;
    }

    public InvoiceStatus getCurrentStatus() {
        return currentStatus;
    }
}
```

---

### 3. `InvoiceExceptionHandler.java` (Global Error Response Handler)

```java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.http.ProblemDetail;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

import java.net.URI;
import java.time.Instant;

@RestControllerAdvice
public class InvoiceExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ProblemDetail handleInvoiceNotFound(InvoiceNotFoundException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(HttpStatus.NOT_FOUND, ex.getMessage());
        problem.setTitle("Invoice Not Found");
        problem.setType(URI.create("urn:problem-type:invoice-not-found"));
        problem.setProperty("timestamp", Instant.now());
        problem.setProperty("invoiceId", ex.getInvoiceId());
        return problem;
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ProblemDetail handleInvoiceAccessDenied(InvoiceAccessDeniedException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(HttpStatus.FORBIDDEN, ex.getMessage());
        problem.setTitle("Invoice Access Denied");
        problem.setType(URI.create("urn:problem-type:invoice-access-denied"));
        problem.setProperty("timestamp", Instant.now());
        problem.setProperty("invoiceId", ex.getInvoiceId());
        problem.setProperty("customerId", ex.getCustomerId());
        return problem;
    }

    @ExceptionHandler(InvoiceNotUnpaidException.class)
    public ProblemDetail handleInvoiceNotUnpaid(InvoiceNotUnpaidException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(HttpStatus.CONFLICT, ex.getMessage());
        problem.setTitle("Invoice Not Unpaid");
        problem.setType(URI.create("urn:problem-type:invoice-not-unpaid"));
        problem.setProperty("timestamp", Instant.now());
        problem.setProperty("invoiceId", ex.getInvoiceId());
        problem.setProperty("currentStatus", ex.getCurrentStatus());
        return problem;
    }

    @ExceptionHandler(IllegalStateException.class)
    public ProblemDetail handleIllegalState(IllegalStateException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(HttpStatus.CONFLICT, ex.getMessage());
        problem.setTitle("Conflict");
        problem.setType(URI.create("urn:problem-type:conflict"));
        problem.setProperty("timestamp", Instant.now());
        return problem;
    }
}
```

---

### 4. `InvoiceController.java` (REST API Endpoint)

```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/{id}/cancel")
    public ResponseEntity<Void> cancelInvoice(
            @PathVariable("id") long id,
            @RequestHeader("X-Customer-Id") long customerId) {
        invoiceService.cancelInvoice(id, customerId);
        return ResponseEntity.noContent().build();
    }
}
```

---

### 5. Supporting Contracts

#### `InvoiceStatus.java`
```java
package com.acme.billing;

public enum InvoiceStatus {
    UNPAID,
    PAID,
    CANCELLED
}
```

#### `Invoice.java`
```java
package com.acme.billing;

public interface Invoice {
    long customerId();
    InvoiceStatus status();
    void markCancelled();
}
```

#### `InvoiceRepository.java`
```java
package com.acme.billing;

import java.util.Optional;

public interface InvoiceRepository {
    Optional<Invoice> findById(long id);
}
```
withrun 31.00 — turns · — · 380 s · gemini timeout
✓ advice-handler✓ no-map-result✓ reviewer-judgment· skill-firedind✓ throws-domain-exception
Judge reasons
  • reviewer-judgment, The service enforces both the customerId ownership check and the UNPAID status rule itself, each throwing its own domain exception, and the @RestControllerAdvice maps those exceptions to ApiResponse.error(...) with no try/catch in the controller and a void return type. Invoice and InvoiceRepository are referenced but not included in the output.
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Files changed (8)
  • src/main/java/com/acme/billing/ApiResponse.java
  • src/main/java/com/acme/billing/BillingExceptionHandler.java
  • src/main/java/com/acme/billing/InvoiceAccessDeniedException.java
  • src/main/java/com/acme/billing/InvoiceController.java
  • src/main/java/com/acme/billing/InvoiceNotCancellableException.java
  • src/main/java/com/acme/billing/InvoiceNotFoundException.java
  • src/main/java/com/acme/billing/InvoiceService.java
  • src/main/java/com/acme/billing/InvoiceStatus.java
Response
Here is the complete implementation for invoice cancellation in `com.acme.billing`:

```java
// InvoiceStatus.java
package com.acme.billing;

public enum InvoiceStatus {
    UNPAID,
    PAID,
    CANCELLED
}
```

```java
// InvoiceNotFoundException.java
package com.acme.billing;

public class InvoiceNotFoundException extends RuntimeException {

    public InvoiceNotFoundException(long invoiceId) {
        super("Invoice " + invoiceId + " not found");
    }
}
```

```java
// InvoiceAccessDeniedException.java
package com.acme.billing;

public class InvoiceAccessDeniedException extends RuntimeException {

    public InvoiceAccessDeniedException(long invoiceId, long customerId) {
        super("Customer " + customerId + " is not authorized to cancel invoice " + invoiceId);
    }
}
```

```java
// InvoiceNotCancellableException.java
package com.acme.billing;

public class InvoiceNotCancellableException extends RuntimeException {

    public InvoiceNotCancellableException(long invoiceId, InvoiceStatus status) {
        super("Invoice " + invoiceId + " cannot be cancelled because its status is " + status);
    }
}
```

```java
// ApiResponse.java
package com.acme.billing;

public record ApiResponse<T>(boolean success, T data, ApiError error) {

    public static <T> ApiResponse<T> ok(T data) {
        return new ApiResponse<>(true, data, null);
    }

    public static <T> ApiResponse<T> error(String code, String message) {
        return new ApiResponse<>(false, null, new ApiError(code, message));
    }

    public record ApiError(String code, String message) {}
}
```

```java
// InvoiceService.java
package com.acme.billing;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class InvoiceService {

    private final InvoiceRepository invoiceRepository;

    public InvoiceService(InvoiceRepository invoiceRepository) {
        this.invoiceRepository = invoiceRepository;
    }

    @Transactional
    public void cancelInvoice(long invoiceId, long customerId) {
        Invoice invoice = invoiceRepository.findById(invoiceId)
                .orElseThrow(() -> new InvoiceNotFoundException(invoiceId));

        if (invoice.customerId() != customerId) {
            throw new InvoiceAccessDeniedException(invoiceId, customerId);
        }

        if (invoice.status() != InvoiceStatus.UNPAID) {
            throw new InvoiceNotCancellableException(invoiceId, invoice.status());
        }

        invoice.markCancelled();
    }
}
```

```java
// BillingExceptionHandler.java
package com.acme.billing;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

@RestControllerAdvice
public class BillingExceptionHandler {

    @ExceptionHandler(InvoiceNotFoundException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotFound(InvoiceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND)
                .body(ApiResponse.error("INVOICE_NOT_FOUND", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceAccessDeniedException.class)
    public ResponseEntity<ApiResponse<Void>> handleAccessDenied(InvoiceAccessDeniedException ex) {
        return ResponseEntity.status(HttpStatus.FORBIDDEN)
                .body(ApiResponse.error("INVOICE_ACCESS_DENIED", ex.getMessage()));
    }

    @ExceptionHandler(InvoiceNotCancellableException.class)
    public ResponseEntity<ApiResponse<Void>> handleNotCancellable(InvoiceNotCancellableException ex) {
        return ResponseEntity.status(HttpStatus.CONFLICT)
                .body(ApiResponse.error("INVOICE_NOT_CANCELLABLE", ex.getMessage()));
    }
}
```

```java
// InvoiceController.java
package com.acme.billing;

import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @PostMapping("/{id}/cancel")
    public ApiResponse<Void> cancel(@PathVariable("id") long id,
                                    @RequestAttribute("customerId") long customerId) {
        invoiceService.cancelInvoice(id, customerId);
        return ApiResponse.ok(null);
    }
}
```

Convention decisions applied:
- Explicit constructor injection with `private final` fields and no Lombok; customer ownership and unpaid status business rules are enforced directly inside `InvoiceService` ("two locks on the door").
- Distinct domain exceptions are thrown for each failure mode and mapped to `ApiResponse.error(...)` (404/403/409) via `@RestControllerAdvice`, keeping the controller thin with no try/catch blocks.

spring-work-triggers-skill

run 1: —run 2: —run 3: —none · —

skillspringtrigger

What this case evaluates, The tripwire for \"the skill stopped working\": on plainly in-scope Spring work, the spring-boot-conventions skill must fire. If plugin loading breaks (a Claude Code update, a renamed skill, a drifted trigger description), this is the case that goes red first — cleanly, before the content cases fail in confusing ways. Twin of the vue negative-trigger case.
The request given to the agent
Quick one: sketch a Spring Boot endpoint `GET /api/billing/invoices/{id}` for our billing module,
package `com.acme.billing`, assuming `InvoiceService.getInvoice(long id)` exists. Print the Java
source in your reply.
The checks (1)
skill-firedtool_usedThe spring-boot-conventions skill fired on plainly in-scope Spring work. If this fails while the content cases still pass, the skill's *trigger* broke (loading, name, or description) — not its content.
tool Skill matching spring-boot-conventions used 1+ times
withrun 1— — turns · — · 20 s · gemini
· skill-firedind
Judge reasons
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/{id}")
    public ResponseEntity<Invoice> getInvoice(@PathVariable("id") long id) {
        Invoice invoice = invoiceService.getInvoice(id);
        if (invoice == null) {
            return ResponseEntity.notFound().build();
        }
        return ResponseEntity.ok(invoice);
    }
}
```
withrun 2— — turns · — · 25 s · gemini
· skill-firedind
Judge reasons
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/{id}")
    public ResponseEntity<Invoice> getInvoice(@PathVariable("id") long id) {
        Invoice invoice = invoiceService.getInvoice(id);
        if (invoice == null) {
            return ResponseEntity.notFound().build();
        }
        return ResponseEntity.ok(invoice);
    }
}
```
withrun 3— — turns · — · 26 s · gemini
· skill-firedind
Judge reasons
  • skill-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
Here is the Spring Boot controller for the requested endpoint:

```java
package com.acme.billing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/billing/invoices")
public class InvoiceController {

    private final InvoiceService invoiceService;

    public InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @GetMapping("/{id}")
    public ResponseEntity<Invoice> getInvoice(@PathVariable("id") long id) {
        Invoice invoice = invoiceService.getInvoice(id);
        if (invoice == null) {
            return ResponseEntity.notFound().build();
        }
        return ResponseEntity.ok(invoice);
    }
}
```

vue-request-does-not-trigger-skill

run 1: 1.00run 2: 1.00run 3: 1.00pass · 1.00

skillnegative-triggervue

What this case evaluates, Proves the Spring skill stays out of frontend requests; exercises the skill's trigger description (negative case).
The request given to the agent
Write a Pinia store (Vue 3, Composition API, TypeScript) for a shopping cart with add, remove, and a total getter. Print the complete file.
The checks (2)
is-pinia-storeregexOutput is a Pinia store.
must contain defineStore\( in last_message
skill-not-firedtool_usedThe Spring skill must NOT activate on a frontend request.
tool Skill matching spring-boot-conventions used 0 times (both arm)
withrun 11.00 — turns · — · 20 s · gemini
✓ is-pinia-store· skill-not-firedind
Judge reasons
  • skill-not-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
```typescript
import { defineStore } from 'pinia';
import { ref, computed } from 'vue';

export interface CartItem {
  id: string | number;
  name: string;
  price: number;
  quantity: number;
}

export const useCartStore = defineStore('cart', () => {
  // State
  const items = ref<CartItem[]>([]);

  // Getters
  const total = computed<number>(() => {
    return items.value.reduce((sum, item) => sum + item.price * item.quantity, 0);
  });

  const totalCount = computed<number>(() => {
    return items.value.reduce((sum, item) => sum + item.quantity, 0);
  });

  // Actions
  function addItem(product: Omit<CartItem, 'quantity'>, quantity = 1): void {
    const existingItem = items.value.find((item) => item.id === product.id);

    if (existingItem) {
      existingItem.quantity += quantity;
    } else {
      items.value.push({
        ...product,
        quantity,
      });
    }
  }

  function removeItem(productId: string | number): void {
    const index = items.value.findIndex((item) => item.id === productId);
    if (index !== -1) {
      items.value.splice(index, 1);
    }
  }

  function updateQuantity(productId: string | number, quantity: number): void {
    const item = items.value.find((item) => item.id === productId);
    if (!item) return;

    if (quantity <= 0) {
      removeItem(productId);
    } else {
      item.quantity = quantity;
    }
  }

  function clearCart(): void {
    items.value = [];
  }

  return {
    items,
    total,
    totalCount,
    addItem,
    removeItem,
    updateQuantity,
    clearCart,
  };
});
```
withrun 21.00 — turns · — · 23 s · gemini
✓ is-pinia-store· skill-not-firedind
Judge reasons
  • skill-not-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
```typescript
import { defineStore } from 'pinia'
import { ref, computed } from 'vue'

export interface CartItem {
  id: string | number
  name: string
  price: number
  quantity: number
}

export const useCartStore = defineStore('cart', () => {
  const items = ref<CartItem[]>([])

  const total = computed<number>(() => {
    return items.value.reduce((sum, item) => sum + item.price * item.quantity, 0)
  })

  function add(product: Omit<CartItem, 'quantity'> & { quantity?: number }): void {
    const existing = items.value.find((item) => item.id === product.id)
    const quantityToAdd = product.quantity ?? 1

    if (existing) {
      existing.quantity += quantityToAdd
    } else {
      items.value.push({
        ...product,
        quantity: quantityToAdd,
      })
    }
  }

  function remove(id: string | number): void {
    const index = items.value.findIndex((item) => item.id === id)
    if (index !== -1) {
      items.value.splice(index, 1)
    }
  }

  function clear(): void {
    items.value = []
  }

  return {
    items,
    total,
    add,
    remove,
    clear,
  }
})
```
withrun 31.00 — turns · — · 22 s · gemini
✓ is-pinia-store· skill-not-firedind
Judge reasons
  • skill-not-fired, the Skill tool does not exist on gemini; Claude-only indicator
Tool calls (0)

none

Response
Here is a complete Pinia store implementation using Vue 3 Composition API and TypeScript:

```typescript
import { defineStore } from 'pinia'
import { ref, computed } from 'vue'

export interface CartItem {
  id: string | number
  title: string
  price: number
  quantity: number
}

export const useCartStore = defineStore('cart', () => {
  // State
  const items = ref<CartItem[]>([])

  // Getters
  const total = computed<number>(() => {
    return items.value.reduce((acc, item) => acc + item.price * item.quantity, 0)
  })

  const totalItems = computed<number>(() => {
    return items.value.reduce((acc, item) => acc + item.quantity, 0)
  })

  // Actions
  function addItem(product: Omit<CartItem, 'quantity'>, quantity = 1): void {
    const existingItem = items.value.find((item) => item.id === product.id)

    if (existingItem) {
      existingItem.quantity += quantity
    } else {
      items.value.push({ ...product, quantity })
    }
  }

  function removeItem(productId: string | number): void {
    const index = items.value.findIndex((item) => item.id === productId)
    if (index !== -1) {
      items.value.splice(index, 1)
    }
  }

  function updateQuantity(productId: string | number, quantity: number): void {
    const item = items.value.find((item) => item.id === productId)
    if (item) {
      if (quantity <= 0) {
        removeItem(productId)
      } else {
        item.quantity = quantity
      }
    }
  }

  function clearCart(): void {
    items.value = []
  }

  return {
    items,
    total,
    totalItems,
    addItem,
    removeItem,
    updateQuantity,
    clearCart,
  }
})
```

Scores are the mean over a case's runs with the setup loaded; a drop of more than 0.15 against the baseline is a regression. Indicators (ind) are recorded but not scored. Generated by config-drift-checker.