Post

Jake Wharton announced yesterday that all the Lysine projects (OkHttp, Okio, Retrofit, SQLDelight…) don’t allow contributors to use LLMs to generate code or write PR descriptions/comments.

32 reposts251 likes

I think this is good for IDK how long, but right now it makes sense. People out there are flooding OSS with sloppy PRs.

Can’t blame him. It’s really tiring and risky to keep up with changes submitted by people who don’t fully understand them, especially in an OSS library. It also feels disrespectful to the maintainers. Is it the best decision for OSS? I’m curious to hear what you think.

I understand it. The health and time of the maintainers is of top importance. That being said, I don't expect this policy to age well. The quality of code and reasoning coming out of LLMs keeps going up. It may not be to JW level yet, but it's surpassed most of us.

For core libraries I feel it is the right direction. API surface and engineering details have to be top notch and from my experience LLMs are not there yet.

Also Fable writing code documentation is horrible using terms like "blast radius", "attack surface" and what not. Am I writing code or making bomb 😬😂

There's no way an OSS project could ever review the volume of changes an army of LLM enabled devs could push at them, so quite understand this. Personally I don't read it as an attempt at wider push back against LLMs in development, but I sense many do.

Given how often LLMs are used to diffuse responsibility (including for your own understanding) rather than any practical use, it should be standard. It won't stop people, but if they are busy making sure their LLM output makes sense because it needs to look right, that is a better situation.

Tbh, this move I heard of recently, to disallow LLM generated content in projects completely, is nonsensical. If the code is well-reviewed, line by line, and found to be good and valid, where's the difference? The problem is not AI per se but its usage.

Opening a PR creates a burden on maintainers who need to do the labor to review it, as you say, line by line. And AI lets "developers" who have no idea what they're doing create giant walls of very plausible looking slop, creating a huge review burden.

It's entirely cost benefit analysis. Are the benefits of the good PRs enabled by AI worth the cost of needing to sift through the slop? Or does it produce better results to just ban LLM generated code so maintainers can direct their finite time and energy elsewhere?