Aaron Roth

Do you have high quality work on the foundations of responsible computing, writ large? This covers everything from work on differential privacy to algorithmic fairness to designing mechanisms and markets to AI alignment. (Submit to FORC!) 1/ responsiblecomputing.org/forc-2027-ca...

Would you like to present it at a focused venue where it will be discussed in the hallways by the leading lights of the field, rather than lost in a cavernous poster session? Consider submitting it to FORC 2027. 2/ responsiblecomputing.org/forc-2027-ca...

An annoying part of the twitter/bluesky discussion of whether LLMs are "only" "next token predictor"/stochastic parrots is that "next token predictor" is contentless - all mappings from inputs to output strings can be factored into a sequence of "next token" distributions.

In this sense you, or any other system that can output text can be written as a "next token predictor". This isn't making any statement about whether brains function similarly to LLMs or not at an internal level, its just a boring syntactic fact about probability distributions.

Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?

But despite being weaker it is not guaranteed to be satisfied. Still, we find some preliminary evidence of non-trivial safety/completeness tradeoffs among real reward models, and these effects come from coallitional rather than individual alignment.

Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?

What about strategic agents? Let both the driver agent and the reviewers be fully strategic and optimizing for their long run discounted payoff. Under the same non-negative span condition, every Nash equilibrium of the game is safe for the Principal--i.e. no worse than baseline.

Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?

There is a remarkably clean characterization of when you can. We start with a simple one-decision model. A system is safe if and only if the Principal's utility is in the non-negative span of the reviewer's utilities. The forward direction is elementary; backwards is LP duality.

Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?

We study this and model both you (the principal) and the reviewer agent(s) as having utility functions you are trying to maximize. The driver agent repeatedly proposes actions. The reviewer agents compare them to a baseline and vote to approve or deny based on perceived utility.

I sometimes see people trying to use complexity theory to argue that building "true" AI is impossible. I find this unreasonably annoying. It requires ignoring what is in front of your face and it ignores that worst-case complexity has been an awful guide in machine learning.

Applying this kind of argument to more general intelligence would have been wrong a decade ago but now requires ignoring the obvious success of AI and particularly the reasoning models from the last few years. A decade ago we could have said it denies computationalism:

I sometimes see people trying to use complexity theory to argue that building "true" AI is impossible. I find this unreasonably annoying. It requires ignoring what is in front of your face and it ignores that worst-case complexity has been an awful guide in machine learning.

For example, the simplest problem in machine learning - learning a linear classifier to minimize classification error --is NP hard even to approximate (to distinguish the ability to get 51% accuracy from 99% accuracy). epubs.siam.org/doi/10.1137/... and thus everything more general too.

We have a new online boosting algorithm which is very efficient and effective. Unlike prior algorithms which maintain many weak learners and ensemble them, we operationalize the "dual view" of boosting. We don't maintain an ensemble. We try to construct an online hard core distribution.

In the end we're putting a bunch of existing pieces together. The fact that multiaccuracy implies weights for a hard core distribution was noticed by @sicapu.bsky.social , Dwork, and Vadhan. Lots of good prior work gave online adversarial algorithms for multiaccuracy and related notions.

We have a new online boosting algorithm which is very efficient and effective. Unlike prior algorithms which maintain many weak learners and ensemble them, we operationalize the "dual view" of boosting. We don't maintain an ensemble. We try to construct an online hard core distribution.

So we get two incomparable boosting guarantees in one. And the algorithm works well, matching or beating the best comparator on real and synthetic datasets. Since we don't maintain an ensemble we run ~ 50x faster than these competitor algorithms as well.

We have a new online boosting algorithm which is very efficient and effective. Unlike prior algorithms which maintain many weak learners and ensemble them, we operationalize the "dual view" of boosting. We don't maintain an ensemble. We try to construct an online hard core distribution.

We make probabilistic forecasts that are multiaccurate with respect to the weak class---i.e. with residuals orthogonal to the weak class. If we have high error, then ex-post transcript reweighting by our own residual error gives a smooth distribution on which no weak learner has non-trivial edge.

Right now we have "problem overhang" - lots of problems we as a community are interested in because smart and charismatic people thought about them and convinced us that these problems are important. So we are happy/interested to see them solved by AI.

Could it be that once AI models exhaust the overhang of existing problems we care about, no new such problems emerge? There will be plenty of true statements with difficult proofs, but we may stop caring about (any of) them.

Right now we have "problem overhang" - lots of problems we as a community are interested in because smart and charismatic people thought about them and convinced us that these problems are important. So we are happy/interested to see them solved by AI.