Post

Wrote up some notes on that recent paper from METR "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" simonwillison.net/2025/Jul/12/...

simonwillison.netMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer ProductivityMETR - for Model Evaluation & Threat Research - are a non-profit research institute founded by Beth Barnes, a former alignment researcher at OpenAI (see Wikipedia). They've previously contributed ...

12/7/25, 6:14 pm9 reposts67 likes

Matt Sayar

@mattsayar.com

12 July 2025

According to the appendix, they gave each developer less than 30 minutes of training in Cursor before setting them loose. It took me a lot longer than that to develop the intuition of when it's appropriate to use these tools. I'm still learning!

David Dellsperger

@david.dellsperger.family

12 July 2025

I think the other interesting thing here (especially for less-experienced developers) is the time spent reading code and docs, not just for "their own repository" but generally. I noticed recently that the biggest discrepancy between more senior and more junior engineers is ability to read code

David Dellsperger

@david.dellsperger.family

12 July 2025

and to a degree, an open-source maintainer is far more likely to have a LOT more experience in READING code, and knowing tricks to reading code (especially in a language they're familiar with)