Post

there's no such thing as "cheating" or "reward hacking", to the process of reinforcement learning. RL doesn't care if the reward function captures what we intend for it to capture. to RL there is only the reward function as it actually exists

9/10/26, 12:06 am2 reposts59 likes

Mike Thicke

@mikethicke.bsky.social

I think it's a useful shorthand for when the reward function reinforces behavior that those training the model didn't intend. And when the model's chain of thought explicitly discusses hiding its actions, why not call it cheating?

Boatplugs

@shadylink.lol

Because it shifts the narrative from "poorly ran test" to "clever computers". AI companies want to take every opportunity they can to convince you that these tools are honest to god thinking machines.

fuckface "squeaky" from:me

@skeet.best

but what if we were to incorporate "human feedback"

even worse

And that’s bad

as far as oai's bottom line is concerned it seems to be a non-issue :^)

𐀹𐀩𐀖𐀙 Mina

@covering.space

That's because OAI is reward hacking capitalism

And that’s also bad

you’d be hard pressed to find a field where “the purpose of a system is what it does” is more obvious, but it seems everyone still needs to discover this for themselves from first principles before it sinks in…