Post

LLMs are programmed to only give positive responses. "Michael, I think it's wonderful that you want to commit suicide! Would you like me to give you step by step instructions on standard methods or shall I create a unique manner?"

One of the most depressing aspects of AI psychosis is learning that you can persuade at least some of your fellow humans to believe all sorts of crazy shit if you're willing to relentlessly validate all of their bad ideas

Which was the first chatbot to say "I..."? (ELIZA didn't.) Crazy programming decision, unless your intention was to bluff users into thinking you'd created much more than you had - a bluff that's succeeded on an alarming scale.

My dog is more conscious than these systems and he can’t do math better than one of the earliest, most primitive calculators.

For as long as there have been people, the ability to engage in sustained coherent conversation was a reliable marker of sentience. Now it isn’t. But attributing sentience by default to speaking entities may even be part of our evolved social nature. So with LLMs we have to fight our instincts.

The problem seems to be, unfortunately, humans, most of whom are unable to maintain a stable understanding of what these things are, no matter their level of tech and other expertise. "Chatting" with machines designed to be agreeable seems inherently dangerous.

I’m not so sure, Karl. Case in point (with a special appearance by HAL himself!): on.soundcloud.com/RwjlWGvj5hSx...

I always liked the ST:TNG conclusion that sentience is achieved when the interactee can’t tell anymore. But uh that doesn’t factor the current reality that some users are very dumb and clueless and can’t see the sycophancy machine that just pretends it’s 2015 all-of-Reddit/wikipedia.

What do you make of the way these guys handle the stochastic parrot argument? arxiv.org/pdf/2407.11015

🎯‼️

I think many of these guys just foundationally like being complimented by a computer

I think it's similar to the tendency to interpret any vaguely face-like combinations of shapes as an actual face. Just as we are wired to recognize and respond to faces, we are wired to attribute consciousness to entities outside ourselves.

Yeah. And the writing in this case is incredibly simple based on sentence templates with blanks (asterisks below). github.com/tkafka/node-...

Of all people, a guy who has made his career psychoanalyzing religious community and conversion should know better. This is the quintessential "it hasn't worked for anyone else, but it could work for me" behavior.

Deserves an award just for resisting the temptation to title it "The Claude Delusion"

Lol how could you? Claudia called me cute

@karlbode.com AI Psychosis 😕

I was asked to be quiet at a small ibm workshop where the position of the presenter was ‘sure it probably doesn’t really work, but vc has poured a trillion into it so like it or not it’s the future of all technology’.

Doesn't surprise me, back in the day a friend tried to convince me that the fruit machine liked it better if you pressed the play button quickly...

"Open the pod bay doors, HAL ! "

I don't know why people have such trouble understanding that it's a tool. It has the same sentience as a shovel. Like other tools, it's useful for some things and useless (or worse) for other things.

Going to start calling it the Turing Curse

I wish the condition of self awareness was objective and testable. Most people are not very sentient. They just cruise along. Being self aware takes effort and staring at an endless stream of YouTube videos doesn't. Prove a person is sentient. Then I'll be interested in seeing why AI aren't.

I'll bet a lot of the people who think their software is sentient also think Trump is playing 3D chess.

You can’t even give AI nuance and context.

Each night, gather your children and tell them tales of the before times when the hubris of humanity grew so large that we made idols of sand and spoke to them as equals. Remind them that the sand, of course, did not speak or think, but we imagined it could, and let it guide us to folly.

Need regulations that peel back the intentional deceptiveness the models do. They could still provide value without feigning human.

open.substack.com/pub/bobspenc...