Post

Frontier LLMs are much better and dramatically faster than I am at finding and fixing bugs. If you are not asking them to debug your code and review your bugs, you are doing it wrong. A thread of Claude wins. (No longer at Google, btw.)

Filippo Valsorda@filippo.abyssdomain.expert

It’s very clear by now that if LLMs are not improving your software quality it’s either a revealed preference (yours or your org’s) for more volume vs more quality, or a skill issue. The level of testing and review they are enabling in the Go cryptography standard library is amazing.

20 reposts191 likes

bit 40 flipping on one darwin box. most flake hunts stop at timing, Claude kept reading those logs until the hardware pattern showed up.

The win is the second pass. Let the model propose the ugly explanation, then ask it to falsify that explanation against the code. That catches more than the first confident patch.

> If you are not asking them to debug your code and review your bugs, you are doing it wrong. I don't ask Claude or any other commercial LLM to do those things because I can't afford to. These tools need to be accessible on personal computers for them to be useful to me and people like me.

I'm currently paying $ 10 a month for an OpenCode Go subscription and using GLM 5.2 at work. It's pretty capable.

That's entirely fair. The good news is that the smaller, open models will inevitably catch up to "good enough for these bugs" and then you won't need the lab models to do this. The lab models may still be ahead, but it will be like phones: good enough is good enough, no need to upgrade.

I keep an eye on the open models and try them occasionally. On my hardware (RTX5070) they're good enough to ask, "does this function do what I think it does". But not good enough (or more likely my hardware isn't good enough) to ask, "find security vulnerabilities in this (large) project".