Vincent Conitzer

Vincent Conitzer

@conitzer.bsky.social

AI professor. Director, Foundations of Cooperative AI Lab at Carnegie Mellon. Head of Technical AI Engagement, Institute for Ethics in AI (Oxford). Author, "Moral AI - And How We Get There." https://www.cs.cmu.edu/~conitzer/

1.5K followers555 following698 posts

Vincent Conitzer
@conitzer.bsky.social

Our new paper led by Ida Mattsson on the game-theoretic limits that an assistant faces when trying to be both helpful and not harmful, when it doesn't know who it is talking to. arxiv.org/abs/2610.27373

arxiv.orgFundamental Impossibility Results in AI Safety: Trading Off between DesiderataIs there a fundamental tradeoff between desiderata for safe AI systems, such as harmlessness and helpfulness? We answer by formalizing the interaction between AI systems and users as a game with three...
Vincent Conitzer
@conitzer.bsky.social

Congratulations to Emin Berker, Ioannis Anagnostides, and Emanuel Tewolde for receiving a WINE 2026 Best Student Paper Award for our paper "What You Earn from What You Learn: Characterization and Complexity of the Value of Information"! arxiv.org/abs/2610.19488

arxiv.orgWhat You Earn from What You Learn: Characterization and Complexity of the Value of InformationWe examine the value of information in extensive-form decision problems: the largest multiplicative gain that a decision maker can obtain from an information refinement, taken over all payoff environm...
View full thread
Vincent Conitzer
@conitzer.bsky.social

(3/4) There are already plenty of things that can be done, from straightforward approaches such as airgapping (as recommended in the letter), to more unusual approaches such as our "shutdown safety valve" technique, to just not deploying the AI at all.

Vincent Conitzer
@conitzer.bsky.social

(4/4) While there are some difficult problems to be worked out, that shouldn't serve as an excuse for companies/other entities not to take commonsense/known precautions that would prevent predictable harm to others. (I do agree with the importance of events being reported.)

Vincent Conitzer
@conitzer.bsky.social

(2/4) This is not poor cyberdefense, but cyberattack. A letter like this should not create the suggestion that it's all just a friendly recommendation while the world figures out what the standards should be.

Vincent Conitzer
@conitzer.bsky.social

`In one run, the model landed on a page referencing an unsolved homicide [...] Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area [...]” ' www.anthropic.com/research/inv...

anthropic.comInvestigating unintended model actions in our evaluations and internal useAnthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Vincent Conitzer
@conitzer.bsky.social

(2/2) (That is still likely to happen for "smaller" results...) The challenges for the field of mathematics (and far beyond) are fundamentally due to the rapid improvement of the models, not due to OpenAI's approach to releasing the results.

Vincent Conitzer
@conitzer.bsky.social

A 3 billion dollar gift to CMU from Ken Griffin, launching CMU Miami and naming the School of Computer Science! www.cmu.edu/news/stories...

cmu.eduCarnegie Mellon University Announces Historic $3 Billion Gift from Ken Griffin, Pioneering a New Model for Higher EducationCarnegie Mellon University (CMU) and Citadel founder and CEO Ken Griffin today announced a historic $3 billion gift to further elevate CMU in Pittsburgh and launch Carnegie Mellon University Miami.