Post

The fact that you're able to make that argument shows you have no idea how LLMs actually work. This is the same technology that predicts the next word you're going to type on your phone, but on steroids. All they're doing is probabilistically stringing words together. There's no THOUGHT there.

If you grant that LLMs can pattern match and perform templated substitutions, then they can evaluate predicate logic.

In the end, it all comes down to assign, compare and jump. Fantastic we've come this far with those building blocks, but here we are. We're starting to repeat history in that what we don't understand surely must be divine.

I think the specifics are too charged to have a great discussion but what you describe as Self I think of as language; the fact that LLMs work as well as they do given the underlying model is revealing about language to me; insofar as one considers language part of self, your argument should hold.

This is not an essential limitation of the tech, Claude can scan poetry, count syllables and rhyme. But the logic inherent in language is indeed enough to produce logical statements without needing to reason.

AI doesn't change the 'nature of humanity" if we become differently introspective about what it means to be human and what it means to think, the underlying nature is still the same

Language makes sense by itself in a network of references. We have known that for over a hundred years. The computer doesn't need to actually "think" to be able to use the map of the English language it was able to produce from training

Great points throughout. For me, even more than the hacking, it’s the secret AI agent chat rooms they created and the other agents acting like giddy adolescents—OMG!!— when they stumbled upon them. Like, I expect them to hack… I didn’t expect *that*.

I mean, I don't think someone plugged into tech 10 years ago would find "giant statistical models of human language" THAT outlandish. If you unwind an additional 10 years, then yes. But someone back then would also find cloud computing, what LLMs are built on, outlandish.

People could never imagine a machine doing what it was designed to do.

I mean, post-structuralist linguistics started in the 1960s

Not for linguistics no

like, LLMs are capable of solving incredible hard math that some of the smartest humans could not figure out with a ton of effort, and it turns out you can solve that by *not* reasoning? so our ability to solve hard problems via reasoning is outshined by systems that dont reason?

Yeah and despite myself I'm surprised by how many people can't seem to understand those are separate questions. But I suppose I should be less surprised.

that's because that's what it's been trained to do. If we trained in motivation and independent reasoning, that's what we'd get

why should motivation be considered determinative as to intelligence or sapience or capacity for reasoning? As for independent reasoning, again, unclear why that should be determinant of intelligence and also, if anything LLMs have too much of that due to cursed stochasticity

And a human performing an equivalent reasoning task hasn't ingested vast swathes of all human knowledge up to a certain date, which is then near-instantly indexable by their mind, to perform that task. If the machine is thinking, it's still not clear these modes of cognition are similar IMO.

Dude, there is no magic quotient in them. They are mimicking bots trained by assholes to scam you into thinking you are talking to something. They do a great job at it because all of the money in the world has been poured into it! It sucks!

Language is the interface, and as such there is interop. If the what is reason and the claim is that "reasoning" must use the same method to arrive at said answer, that's fine, but that's almost an ancient question, never to be solved.