Post

I gave Opus 5.5, considered the best model these days, a medium-sized task. Even after a full adversarial review and simplify loop to convergence, it produced code that worked but was garbage, with limited use of newtypes and representable invalid states everywhere

I prompted it to fix that, giving a couple of hand written examples, and it did. But then later in the same session it forgot about that and added more such representable states. I refuse to lower my standards to what these things write by default

Code that works at a point in time is mostly uninteresting. What matters is building structures to ensure code stays correct over time. That to me is the essence of engineering in software

40 reposts214 likes

It feels industrially oriented to me, which is a virtue, is certainty oriented. I do hope we respect the other natures too though. The Engelbart Augment Intellect vein to me is much more about ad hoc discovery, connection, serendipity. Malleability.

We can engineer for possibility too. Engineering *is* very much the predictable, the measured. I give huge ground to that claim. But it also sometimes too orients itself beyond the measured careful known norms. Sometimes. Not often.

I mean software engineering as a discipline specifically, as distinct from the much larger context of programming (which does include a lot of what you said)

I'm 99.9% in agreement on engineering and 99.9% in agreement that most of the rest can be something else, but I do think that protocols engineering as maybe an example, that systems engineering is often this trying to engineer beyond the edge of known space, engineer for possibility.

I need to reflect on this a little more, maybe try to open to a 99% agreement, by thinking this over. But reflexively I do not like yielding so much, narrowing down engineering so strongly. But I also see that overall: that's 99% the case

Look this is sometimes true but also most of nextest's core architecture is basically the same as it was day 1. Plenty has changed around it but this page, extracted from the first design doc I wrote for it, has not nexte.st/docs/design/...

Plenty of value in prototyping and trying out a few options before committing. LLMs can be great assistants at that. But this "throw the first 9 versions away" attitude has always been strange to me in code. There is so much value in committing and iterating on something

People are not reliable, either. We build processes to produce reliable outcomes from unreliable components.

The human body works surprisingly well, at least compared to most of our systems. Self-replicating, self-repairing, for most an amazing uptime on complex systems like the liver and heart. It has all sorts of different chemical pathways to success. Admittedly it took a while to develop.

Building 10 times only works if you learn from each previous time. LLMs don't learn. The quote is attributing a special value to the number of times when the real value is in the learning