Alexander Terenin

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

So this sums up the work! If you followed along, thanks for the interest! I think you'd agree that "Bayesian Algorithms for Adversarial Online Learning: from Finite to Infinite Action Spaces" is a much better title than before. The old one was much harder to pronounce.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

We prove that the Bayesian approach works in this setting too. To achieve this, we develop a new probabilistic analysis of correlated Gaussian follow-the-perturbed-leader algorithms, of which ours is a special case. This has been an open challenge in the area.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

The first one is the classical discrete setting where standard algorithms such as exponential weights are studied. You can use a Gaussian prior which is independent across actions.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

This approach appears to not make any sense: the Bayesian model is completely fake. We're pretending to know a distribution for how the adversary will act in the future. But, in reality, they can do anything. And yet... we show that this works!

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

One observation about adversarial online learning is that it appears to have nothing to do with Bayesian learning. There is a two-player zero-sum game, not a joint probability distribution. So you can't just solve it by applying Bayes' Rule. Or can you?

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

Online learning is therefore a good model for learning to explore by taking random actions. In contrast to other approaches to resolving explore-exploit tradeoffs such as upper confidence bounds which produce purely deterministic strategies.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

Specifically, their goal is to minimize regret R(p,q) = E_{x_t~p_t, y_t~q_t} \sup_{x\in X} \sum_{t=1}^T y_t(x) - \sum_{t=1}^T y_t(x_t). Meaning, the learner compares the sum of their rewards y_t(x_t) with the sum of y_t(x) for the best possible single non-time-dependent x.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

"Adversarial Online Learning" refers to a two-player zero-sum repeated game between a learner and adversary. At each time point: - The learner chooses a distribution of predictions p_t over an action space X. - The adversary chooses a reward function y_t : X -> R.

Paper update: our recent work on Thompson sampling has a shiny new - and I hope much better - name! This new name does much better job of emphasizing what we actually do. Joint work with Jeff Negrea. Thread below!

At ICML, we're presenting a paper on uncertainty-aware surface reconstruction! Compared to previous approaches, we are able to completely remove the need for recursive linear solves for reconstruction and interpolation, using geometric GP machinery. Check it out!

This week’s virtual seminar on Bayesian Decision-making and Uncertainty is happening now! Noémie Jaquier (KTH Royal Institute of Technology) On Riemannian Latent Variable Models and Pullback Metrics Livestream link: www.youtube.com/watch?v=61Be...

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

We obtain a very simple condition which relates the prior covariance kernel with the adversary's function class in an easy-to-verify way that works in the bounded Lipschitz case. I am very interested in extensions to more-general smoothness classes, and have ideas. Stay tuned!

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

Specifically, we have a novel probabilistic argument which bounds Hessian-type terms which appear in the regret analysis of Gaussian follow-the-perturbed-leader algorithms, of which Thompson sampling is a special case. Our argument works even with correlations!

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

Using this viewpoint, we introduce a Thompson sampling variant with a Gaussian process prior, and prove an adversarial guarantee against a bounded Lipschitz adversary. So what's new? We now have an analysis that works in any dimension!

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

Let's first remember what this paper does: it gives a new way to think about designing algorithms for the so-called general-action-space online learning game - where a learner makes a prediction, and an adversary responds with a reward function in some class of possible rewards.

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

But first, the previous announcement of this work, which describes in more detail what is going on and why I think it's important: bsky.app/profile/avt.... This work is joint with Jeff Negrea, who has been a great pleasure to collaborate with!

This week’s virtual seminar on Bayesian Decision-making and Uncertainty is happening now! Marvin Pförtner (University of Tübingen) Computation-Aware Kalman Filtering and Smoothing YouTube livestream: www.youtube.com/watch?v=0tG2...

This week’s virtual seminar on Bayesian Decision-making and Uncertainty is happening now! Yingzhen Li (Imperial College London) On "Modernising" Sparse Gaussian Processes YouTube livestream: www.youtube.com/watch?v=VbGW...

We’d like to announce next week’s virtual seminar on Bayesian Decision-making and Uncertainty! Yingzhen Li (Imperial College London) On Modernising Sparse Gaussian Processes Sign-up link below!

This week’s virtual seminar on Bayesian Decision-making and Uncertainty is happening now! Sebastian Ament (Meta) Unexpected Improvements to Expected Improvement for Bayesian Optimization YouTube livestream: www.youtube.com/watch?v=B71O...