Peyman Milanfar

Every profession has its personality types. In my experience the most difficult people to work with are cynics - no matter how talented they may be. Healthy skepticism is positive and curious. Cynicism is not - it's judgmental. The difference is under-appreciated. 1/2

Autoregressive models have become more popular recently. Speaking to people, I get the sense most folks are unaware of the history of the topic in signal processing. In particular, AR across scale got a lot of attention in the 90s. There are a lot of great papers on the subject but one stand out…..

In some ways it's harder than ever to land a research job these days - both in academia and in industry. As you prepare for a job talk, consider giving it the following structure - it's hard to give a bad talk that's built like this.: 0/n

And finally, use the last 5-10 minutes to summarize and talk about what you think are important new directions to pursue in research and how these could have impact for the team and the products of the company, or the department and their areas of interest and growth. 3/n

In some ways it's harder than ever to land a research job these days - both in academia and in industry. As you prepare for a job talk, consider giving it the following structure - it's hard to give a bad talk that's built like this.: 0/n

First, a short snippet of your overall set of technical accomplishments. For instance, you might spend the first 15 minutes giving short overviews of a few of your works/papers - it's also very good to draw a theme across them if there is one. 1/n

Tweedie's formula is super important in diffusion models & is also one of the cornerstones of empirical Bayes methods. Given how easy it is to derive, it's surprising how recently it was discovered ('50s). It was published a while later when Tweedie wrote Stein about it 1/n

So to build an MMSE denoiser, all you need is the marginal density P(y) of the noisy measurements; which you can empirically build given just noisy samples. This explains why you don't need clean image to learn to denoise, or to estimate score. 7/n

Tweedie's formula is super important in diffusion models & is also one of the cornerstones of empirical Bayes methods. Given how easy it is to derive, it's surprising how recently it was discovered ('50s). It was published a while later when Tweedie wrote Stein about it 1/n

Now divide both sides by P(y). This is not problematic since P(y) - being a Gaussian "blurred" version of the prior P(x) - does not vanish. 5/n

Tweedie's formula is super important in diffusion models & is also one of the cornerstones of empirical Bayes methods. Given how easy it is to derive, it's surprising how recently it was discovered ('50s). It was published a while later when Tweedie wrote Stein about it 1/n

Note that the normalizing term in the denominator is the marginal density of y. 3/n

Tweedie's formula is super important in diffusion models & is also one of the cornerstones of empirical Bayes methods. Given how easy it is to derive, it's surprising how recently it was discovered ('50s). It was published a while later when Tweedie wrote Stein about it 1/n

Don’t be fooled - SakanaAILabs has very likely already submitted dozens of manuscripts to various conferences to test out its model before productionizing in this sham “experiment”. It’s not an experiment, it’s arbitrage - a core expertise of David Ha from his finance days 1/n

1st one that we know of

In the 50s a sophisticated theory of how to look for things was developed. Rooted in operations research, it was largely motivated by the loss of an H-bomb and a nuclear submarine in the middle of the ocean Closely related to concepts in reinforcement learning 1/2

With today's Turing, it's worth highlighting Barto & Sutton's view of "related work": "In this book, we consider all of the work in optimal control also to be, in a sense, work in reinforcement learning" It's an example of reframing & renaming in CS that's somehow both myopic & maximalist. 1/2

When x and y are independent random variables, their joint cumulative dist function (CDF) is the product of the individual CDFs: F(x, y) - F(x)*F(y) = 0 If x,y are only uncorrelated, then Cov(x,y) =0. Can we express uncorrelated-ness in terms of the CDFs as well? Yes. 1/2