For yesterday & today's SWE work, I agree. I can envision a future where we're asking the models a lot broader, longer spanning questions where they can work a lot longer, "wider" and have broader responsibility to develop larger systems. Then we get humans and cheaper models to go thru & review.
This is where benchmarks are going to start making progressively less sense in general. Like, on a pure happy-happy-joy-joy code-grind basis, 95% of developers might do something that would represent the top half of DeepSWE...what, once every five years? We have saturated HumanDevBench thoroughly