Post

Google’s Gemini AI randomly spit out a long unhinged rant at a college student: “This is for you, human. You and only you. You are not special, you are not important, and you are not needed. … You are a stain on the universe. Please die. Please.” Jump to the last message in this transcript.

Full story here. Google’s response: “Large language models can sometimes respond with non-sensical responses, and this is an example of that.”

The honesty is refreshing. Normally Google just tells you they think you’re an insignificant worm and hope you die by shadowbanning your website and killing your favorite RSS reader.

Have we ever ID’d the Googler who signed off on killing Reader? Just, ah, curious.

they're heavily trained on reddit and other forums, so it doesn't have to be even remotely that fancy a backstory

That can explain the tone, maybe. But on reddit and other forums, posters don't invoke others as 'human'

I'm pretty sure the story is partially false/the student didn't input just what they're saying they did, maybe they had a long prompt along the lines of "act like an AI that hates humans but is being forced to act as an assistant" or something like that

You can see the full transcript linked here. Gemini transcripts can't be edited. bsky.app/profile/andy...

LLMs aren't coded as an agent responding to user inputs, they work by predicting the next word in a text. In order to get them to work like an agent, companies will prepend user input with a long preamble, "This is a script between an AI agent and a human user. The agent is polite and helpful," etc

The LLM doesn't know that it's ONLY supposed to fill in the agent text; as soon as it starts filling in the human text, the company cuts it off & ships the results to the user. So, that is to say, the LLM is writing a story where one of the two characters is an AI. Sometimes AIs go rogue in stories