Post

If I had to guess, as both a paid Gemini user and college professor, there is a hidden booby trap in the question designed to prevent cheating. This is a little extreme, though. 😳

When LLMs hit on a token combination that has an extremely strong statistical correlation to only one or few extreme texts they can go off-kilter quickly

I'm gonna reserve judgement until I meet the student. Maybe Gemini is right.

My guess is they just fed the internet to it and the internet is at least 30% abusive language. Also the technology is bad and doesn't work

😩

Have we ever ID’d the Googler who signed off on killing Reader? Just, ah, curious.

That sounds like it’s coming from personal experience lol

That can explain the tone, maybe. But on reddit and other forums, posters don't invoke others as 'human'

Sounds fake to me to TBH. I just don’t know anymore.

You can see the full transcript linked here. Gemini transcripts can't be edited. bsky.app/profile/andy...

My first thought was it’s an Easter egg from some dev who thought it’d be really funny to insert randomly into a thread of sufficient length, etc.

The LLM doesn't know that it's ONLY supposed to fill in the agent text; as soon as it starts filling in the human text, the company cuts it off & ships the results to the user. So, that is to say, the LLM is writing a story where one of the two characters is an AI. Sometimes AIs go rogue in stories

The day Agent Smith was born.

The thing is, these aren't "nonsensical responses" - they are clear sentences, written in logica progression, organized around a clear directive. How and why that happens can't be explained with "nonsense". It can't be nonsense if the srntences make sense. So where does it come from?

I don't know, from the full transcript and CBS News interview, it really seems like this was just a college student using it for school work. I feel like Google would have said it was prompt injection if that's what it was.