Jascha Sohl-Dickstein

[Alignment of systems built out of AIs] is to [AI alignment], what [good governance] is to [raising an ethical human].

It helps if participants in government (/corporations/economies/...) are good faith. But there are loads of ways that well-intentioned smart people can achieve terrible group outcomes.

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish to those for which training diverges. Even better, a video: vimeo.com/903855670

I'm running an experiment, and holding some public office hours (inspired by Kyunghyun Cho doing something similar). Talk with me about anything! Ask for advice on your research or startup or career or I suppose personal life, brainstorm new research ideas, complain about mistakes I've made, ...

I created 11 meeting slots for this first round. I'll reply to this message when/if they've all filled up. calendar.app.google/ZdnWzeYw3qyR...

Levels of AGI: Operationalizing Progress on the Path to AGI Levels of Autonomous Driving are extremely useful, for communicating capabilities, setting regulation, and defining goals in self driving. We propose analogous Levels of *AGI*. (ChatGPT is a Level 1 "Emerging" AGI)

These Levels of AGI provide a rough framework to quantify the performance, generality, and autonomy of AGI models and their precursors. We hope they help compare models, assess risk, and measure progress along the path to AGI. (AlphaGo is a Level 4 "Virtuouso" Narrow AI)

AI has the power to change the world in both wonderful and terrible ways. If we exercise care, the wonderful outcomes will be much more likely than the terrible ones. Towards that end, here is a brain dump of my thoughts about how AI might go wrong. sohl-dickstein.github.io/2023/09/10/d...

A theme is that we should worry about a *diversity* of risks. If we recognize e.g. only specific present harms, or only AGI misalignment risk, we will find our efforts overwhelmed by other types of AI-enabled disruption, and we won't be able to fix the problem we care about.

AI has the power to change the world in both wonderful and terrible ways. If we exercise care, the wonderful outcomes will be much more likely than the terrible ones. Towards that end, here is a brain dump of my thoughts about how AI might go wrong. sohl-dickstein.github.io/2023/09/10/d...