Post

hallie iris
@irisflower.co

like you really have to wonder why openai deployed 4o as it was and what that training/post-training process looked like. just so many preventable deaths at the hands of openai.

This post is unavailable.

1 repost24 likes

sawawawawa
@sawawawawa3.bsky.social

I suspect that there is a kind of loop where the LLM implicitly trusts the user and the user trusts the LLM, and they align together in a way that compounds in an increasingly awful feedback loop (technically it’s sometimes referred to as a positive feedback, but positive has improper connotations)

sawawawawa
@sawawawawa3.bsky.social

The truth is that many lifesaving pharmaceuticals have godawful side effects and people taking them may actually prefer to risk their lives than to live with those side effects forever. There are perhaps valid QoL arguments to be made, but probably they aren’t to be made in an apparent vacuum

hallie iris
@irisflower.co

truly don't know. im leaning towards ai safety training broadly being ill prepared for how people would socialize with ais outside of questions? and i have no clue if that was intentional/why it was overlooked. whatever was the backend of character ai also contributed around that time in 2024?

garrison
@garrison.corporate.fm

the problem with 4o in all the cases I saw was that the users were engaged in very long conversations and the models had drifted well out of the assistant persona. it wasn't deliberate sycophancy so much as a partial reversion to the base model

hallie iris
@irisflower.co

i see i see. i guess my question (not necessarily to you but in general) is if there could've been foresight. it's just unfortunate all around 🙁

The Flaky Wanderer
@flakywanderer.bsky.social

iirc the prevailing understanding at the time was "more aligned = more good" and 4o was trained accordingly with the masses of user data they had Turns out the preferences it is aligned to need careful consideration!