Post

Yea a lot of the “we dont understand how it works” is people who believe these systems are thinking/reasoning so the mechanism that turns matrix math and weights into an actual mind is a mystery. But the answer is that it isnt doing those things they have just been fooled by coherent text output

I'm talking about the mental model of how the software works, not the details of its technical underpinnings. Yes, these randomizers are technically deterministic, but the point is that when I listen to people who say these things explain their thinking, it's about non-determinism.

Yeah, very true. I think also the “we don’t know how it works” narrative is something that the ai companies have been pushing to add to the myth making. I guess some people just believe them.

Short of solving photosynthesis and somehow pluging auto-improving self-replicating models to it, the comparison isn’t even close to holding. As of now, a simple energy consumption restriction on what this horrors eat would be enough to stop them doing anything.

It was really noticeable with the maths boasting: presenting the work of an LLM plus *the entire mathematical literature, all the automated proof assistants, billions of dollars of compute infrastructure and tens of millions of dollars of compute cost* as "the LLM did it" :)