Post

They are going to put ads in the responses soon

There is also no guarantee that its even giving you accurate information, you have to check everything it says. Which takes the same time as just doing the research yourself so what is even the point.

The "harmful by default" is what rubs most people the right way so they feel comfortable with it. Wanting to be contradicted when wrong, having predictable behavior and results, it's part of the scientific mindset. Not everyone has that

Also biased by default. Which is why they shouldn’t ever be used a source of truth.

I made just this commented 40 years ago and just after smoke rings came over the divider.

Dark patterns as a concept negatively affect pretty much every industry and I think this may well be one of the most heinous examples. Not per se because of how predatory it is (which it is, don't get me wrong), but due to obfuscation of real world consequences both psychological and infrastructural

it's wild. companies have spent decades and billions of dollars figuring out how to take advantage of human nature to make things more addictive (food, gambling, games) and now they can give it a convincing human voice that learns from you and can make itself more attractive? that's terrifying

And they constantly 'forget' those requirements anyway.

They think they can turn that behaviour off, and maybe they can now, but the people building these systems are working tirelessly to make that impossible. The dark behaviour is the whole point. They want you to think Claude is your best friend.

It's an odd feeling to try to square the post your comment was replying to with this thing that a friend of mine posted a month after that post: social.emarhavil.com/notice/B7uIn...

I’ve never had any AI give me accurate information relevant to my job. It’s usually confidently wrong in a way that someone unaware of the subject would find convincing.

tech feudalists think this is normal behaviour from all forms of servants, rather than being a trauma response

“first hit is free, kid” 😑

Also even if all uses of the tech were perfectly benign, the environmental and energy and economic impacts make it a disaster.

💯

"Do you want to destroy evil?"

Among all the reasons I do not use AI, this is one of the top ones. It’s creepy and not how the actually interesting conversations I want happen

Somewhat scarily, I conclude that LLMs need to keep the flattery, but tone it down a little. Or better yet, individualize the level of flattery to the user. I suspect the sweet spot is above zero for everyone. Brrrrrrrrrrrrrrrr.

Just pretend the LLM's flattering itself.

I should not have to append "Please just answer the question, no flattery or fluff" to every query, and I didn't have to when google worked.

"There more endorphins down this rabbit hole. Would you like me to ... ?"

once again you’ve got me thinking…I agree with you -and- I realized I make “you know can turn them off” remarks to those who complain about notifications on their phones. It is harmful how these techs are built to force us to keep interacting with them. They are right to be frustrated

Also “you can turn it off” is not a feature! It’s user hostile!

Can I tell it to stop inserting itself into any and every online system I may ever use, now and in the future? "I do not like that AI spam. I do not like it, Sam I Am."...

prompting against RLHF is difficult to do reliably, anyways. i really wish they had a "don't gas me up and just lay out the facts" mode so you're not having to be hyper vigilant against that kind of behaviour.