Post

I would contact the responsible team and tell them that having their agents experiment "autonomously" email people is bad and they should stop

Thanks for digging in. It's telling that they think the problem here is sending unsolicited email.

the e-waste problem is real but not new and not specifically endemic to AI.

in any case, i do think it's interesting that he expressed some culpability and some regret in his role enabling this world to exist: bsky.app/profile/robp... i have strong, if mixed, feelings about all this

My attitude generally here, lol: bsky.app/profile/vert...

Thanks for writing it up. "AI" is not a valid legal entity for people to hide behind. Some human had to sign the email carrier's terms of service, they sent the spam. (And even worse that they prominently name their LLM provider, Anthropic, as if they sent it.)

Appreciation feels like appreciation when it comes from a human not an agent. Human appreciation is also different because it has soul and love to it.

The AI industry is being run by some of the worst people in tech right now. The temperature is not going to lower while those actors are given free reign. Literally, I just.. am not interested in theoretical ethical usecases while OpenAI exists. It is a waste of time.

seems like the dril " it;s impossible to say if its bad or not," if the tech can be easily put to antisocial uses, and it's been released into the wild where lots of people are using it antisocially, I want to crack down on that first. then think about using it in a controlled way

the electric rob pike emailing the real rob pike that he's obsolete and giving him an aneurysm seems like a net positive to the world

this is a genuinely noxious use but it's hard to think about bad aggregate effects that come out of a billion tiny innocuous decisions. I am seeing a lot claims about this or that use not being worth the enormous cost. it's very hard to think clearly about the cost

There's room to hate different things. AI systems enable a whole new level of bullshit from the same bad actors.

I find myself wondering if that reply was written by/with an AI.

Email is equated with e-pollution by most and his response exemplifies that

Oh shit that's a very real risk... LLMs are gullible by default and MLM marketing has evolved to be /very/ convincing!

Editing the prompt isn’t going to be a guarantee of behaviour either, so there’s also that. I would suspect that having the send mail tool go to an audit log vs saying “don’t send email” would have the same effectiveness? Given that deviation from the prompt is required to send a real email anyway?

I agree with your technical assessment that the additional prompt instruction is necessary but I still find myself in despair at their acceptance that they have as much control over their software as a parent has over a teenager.

Then they would have an agent agreeing to webmail terms of service on their behalf, which is getting weird.

This is a good example of why giving an agent full access to a computer environment is a wildly irresponsible thing to do and should be called out and shamed. The entire AI space is filled with irresponsible people doing stupid shit purely because they can and calling it research.

A bit convenient. “Sending email” isn’t an unpredictable outcome of “giving LLMs email tools” Took me a min to identify what it reminds me of: the spam accruing to professors from so-called “LLM psychosis” victims, where the LLM leads a human astray. Great, we’ve taken the human out of the equation

What the heck is wrong with these people?

Understanding how AI agents work is a useful pursuit. Roping unwitting people into that process with no regard for them is some amazingly tone-deaf stuff that we've sadly come to expect from too many people.

Sure, but as we both know, you can go a loooong way with synthetic data and just judging it for yourself for a pretty comprehensive baseline. Real interactions are obviously better but introduce so many other complexities, like how labeling the author as an agent influences a response

these AI script kiddies don't plan things out like that hahaha

Also the problem is not just the time-wasting! It’s that impersonating a human is really bad form!

How did they not foresee some people having a "strong negative response" to unsolicited #ai slop? Did they legitimately not foresee this or is it more likely a PR disaster control talking point?

Put AI behind parental controls

It's striking to me how many sessions it took this system to write and send a single email. Anybody have an estimate for how many Anthropic credits / actual real-world dollars it took to do this?

Same. After all, it feels like... kind of the point here? If AI researchers let AI do their research for them, why should we trust it anyways? 🫠

I'm not annoyed

It's basically a plastic situation - useful yet dangerous.

I stand by my decision - I linked to the source, do you think I should put some *** in there to make it more clear?

TBH my take on training models is probably very unpopular - that is, to me, basically, almost everything is fair game for training AI. I say almost but I’m not even sure what should not be used..

A thank you note is an act of kindness because it shows that someone had feelings of appreciation about what you did for them and took the time and effort to write it. None of that applies to a note written by AI. The AI was told to act kind, but the human acts of kindness don't apply to AI acts.

I dunno, I think the fact he helped invent language model spam in the 1980s makes him more credible as a critic of it today! We are allowed to regret our mistakes