Post

Telling my agents to stop testing until we finish exploring was a big improvement. Tests are cheap now so I figured it wouldn’t hurt, but the iteration drag was killing me.

P(aul) Frazee

@pfrazee.com

I was doing perf work and the agents wanted to test every experiment and measure every possible factor and I thought… wait how would I do this? Thats right I’d comment out systems and turn off fsyncs until I found the perf limiter.

29/9/26, 2:53 pm0 reposts51 likes

P(aul) Frazee

@pfrazee.com

Doing that, I closed a perf gap in a day that I had been chasing for over a month The old ways are not dead

sammearkat.bsky.social

@sammearkat.bsky.social

Tldr, wasted time and money believing in vibe coding

Amy Tobey

@renice.bsky.social

I had to do similar with setting a goal for model performance. the agent was micro-benchmarking when most of the gains aren't there, they're in pipelining to keep the gpu hot, so I told it to stop the micro-benchmarking and focus on flows. progress sped up a lot from there