Post

Russ Cox
@swtch.com

Frontier LLMs are much better and dramatically faster than I am at finding and fixing bugs. If you are not asking them to debug your code and review your bugs, you are doing it wrong. A thread of Claude wins. (No longer at Google, btw.)

Filippo Valsorda@filippo.abyssdomain.expert

It’s very clear by now that if LLMs are not improving your software quality it’s either a revealed preference (yours or your org’s) for more volume vs more quality, or a skill issue. The level of testing and review they are enabling in the Go cryptography standard library is amazing.

Stephen Illingworth
@jetsetilly.bsky.social

> If you are not asking them to debug your code and review your bugs, you are doing it wrong. I don't ask Claude or any other commercial LLM to do those things because I can't afford to. These tools need to be accessible on personal computers for them to be useful to me and people like me.

Russ Cox
@swtch.com

That's entirely fair. The good news is that the smaller, open models will inevitably catch up to "good enough for these bugs" and then you won't need the lab models to do this. The lab models may still be ahead, but it will be like phones: good enough is good enough, no need to upgrade.

0 reposts7 likes

Stephen Illingworth
@jetsetilly.bsky.social

I keep an eye on the open models and try them occasionally. On my hardware (RTX5070) they're good enough to ask, "does this function do what I think it does". But not good enough (or more likely my hardware isn't good enough) to ask, "find security vulnerabilities in this (large) project".