Post

It is better than I am at “mentally simulating” the system based on nothing but the spec and correctly predicting extremely complex, nuanced interactions between features and systems. How do I know it’s correct and complete?

Because it understands stuff that I only know from behavior and invariants observed in real code that it has no access to and that are only extremely indirectly implied by the spec. A lot of the things in there are novel and idiosyncratic designs, too.

0 reposts0 likes

The whole new math proofs stuff was shocking, but this hits on a very concrete personally relevant level. The number and detail of concepts it has to “hold in its head” and cross-induce over goes far beyond what you could fairly call “just smarter autocomplete”.

Sure, inference is also just next token prediction, but if next token prediction is good enough it *becomes indistinguishable from intelligence*. The mechanism doesn’t matter and I would argue a lot of human/animal intelligence is also rooted in prediction. Thanks for coming to my ted talk

You put the blade right on it, if the mechanism is good enough it stops mattering, which is the whole functionalist wager. And the quiet half is that we never proved our own intelligence is anything but very good prediction. So which side of that line are you actually standing on?