Post

Worth noting those runs had permission prompts off. On my repos the scope line in the prompt is a hint, not a control. The control is what the agent can touch: write access to the files in the ticket, read-only on the rest. Spotting a bug elsewhere gets a comment, not a commit.

I tell coding agents what to change and what to leave alone. If they spot another problem, I want them to report it, not fix it. Inferring intent shouldn't mean expanding the job but sometime that occurs.

fine print in the piece: the May study ran Claude Code with --dangerously-skip-permissions, so Sonnet 4.6 going from 3.9% to 15.8% out-of-scope is with permission prompts off

Scope creep is the part I'd most want fixed. An agent that guesses intent well is nice, but one that stops and asks before touching files outside the task is the one I'd actually trust.

Spotting a problem and having permission to fix it are two separate things. I'd write down what an agent may change before it starts and have it ask before touching anything else, like the example in the piece where it offers to fix similar bugs. More room can come with a track record.

Scope creep shows up most clearly in tool calls, the log of what ran. Was the study's count taken from that log?