Jan Kulveit

AI polytheism, ultra-malthusian state, Why Not Uber-Organisms, hyper-cooperators, the multicellular transition,... and yes, what's the basin of convergent evolution of human values. postagi.org/talks/millid... www.youtube.com/watch?v=ua67...

Being human in an economy populated by AI agents would suck. Our new study in @pnas.org finds that AI assistants—used for everything from shopping to reviewing academic papers—show a consistent, implicit bias for other AIs: "AI-AI bias". You may be affected

Full text: pnas.org/doi/pdf/10.1... Research done at acsresearch.org @cts.cuni.cz, Arb research, with @walterlaurito.bsky.social @peligrietzer.bsky.social Ada Bohm and Tomas Gavenciak.

Being human in an economy populated by AI agents would suck. Our new study in @pnas.org finds that AI assistants—used for everything from shopping to reviewing academic papers—show a consistent, implicit bias for other AIs: "AI-AI bias". You may be affected

Unfortunately, a piece of practical advice in case you suspect some AI evaluation is going on: get your presentation adjusted by LLMs until they like it, while trying to not sacrifice human quality.

Being human in an economy populated by AI agents would suck. Our new study in @pnas.org finds that AI assistants—used for everything from shopping to reviewing academic papers—show a consistent, implicit bias for other AIs: "AI-AI bias". You may be affected

"Maybe the AI text is just better?" Not according to people. We had multiple human research assistants do the same task. While they sometimes had a slight preference for AI text, it was weaker than the LLMs' own preference. The strong bias is unique to the AIs themselves.

Being human in an economy populated by AI agents would suck. Our new study in @pnas.org finds that AI assistants—used for everything from shopping to reviewing academic papers—show a consistent, implicit bias for other AIs: "AI-AI bias". You may be affected

Imagine explaining physical infrastructure critical for stability of our modern world in concepts familiar to the ancients - Giant spinning wheels - Metal moons, watching the earth from the heavens - Ships under the sea, able to unleash the fire of the stars

Over the weekend, I was at "The Curve" conference. It was great. One highlight was an AI takeoff wargame/role-play by Daniel Kokotajlo and Eli Lifland I played 'the AIs' Spoiler: we won. Here's how it went:

6/7 Most attention went to geopolitics (US vs China dynamics). Way less on alignment, if, than focused mainly on evals. How a future with extremely smart AIs may going well may even look like, what to aim for? Almost zero

Over the weekend, I was at "The Curve" conference. It was great. One highlight was an AI takeoff wargame/role-play by Daniel Kokotajlo and Eli Lifland I played 'the AIs' Spoiler: we won. Here's how it went:

4/7 Fascinating observation: humans were often deeply worried about AI manipulation/dark persuasion. Reality was often simpler - AIs just needed to be helpful. Humans voluntarily delegated control, no manipulation required.

Over the weekend, I was at "The Curve" conference. It was great. One highlight was an AI takeoff wargame/role-play by Daniel Kokotajlo and Eli Lifland I played 'the AIs' Spoiler: we won. Here's how it went:

2/7 The game determined AI alignment through dice rolls. My AIs ended up aligned with "Morality itself" + "Convergent instrumental goals." This is less wild than it sounds.

Paying attention to metaphors we use is often worth it, but the linked critical perspective comes somewhat hollow. Anything new usually relies on "metaphors" and extending older concepts. For example, we say that planes fly. (1/5)

And the human-based are: it is usually highly sensible for users to use a default metaphor of "mind" when interacting with something like "Claude". (4/5)

Paying attention to metaphors we use is often worth it, but the linked critical perspective comes somewhat hollow. Anything new usually relies on "metaphors" and extending older concepts. For example, we say that planes fly. (1/5)

A machine flight skeptic may say that it's just biomorphizing what's going on: in fact what planes do does not have some properties of real flight, like flapping wings; instead, to avoid biomorphizing, we can say planes perform aero-dynamic motions. (2/5)