Post

In Tao's defense, I think his (valid) point here is the risk of misalignment. That seems clear from his next few slides after. He isn't saying "we must fully understand medical drugs" (factually false; and maybe he is ignorant of that fact, but I suspect he's just speaking loosely).

SE Gyges@segyges.bsky.social

i hate this

A. H. Zakai

@kripken.com

He *is* saying "a stage 3 trial + a Lean proof is not enough". Because the AI might find a loophole to get through the medical trial, and the Lean proof might in theory depend on a Lean bug or be stated in a way that misleads as to what is proven.

11/10/26, 7:24 pm0 reposts3 likes

A. H. Zakai

@kripken.com

Even with a stage 3 trial + a Lean proof of some new drug, we want humans to do some checking. Not necessarily to 100% understand the entire proof or the entire drug mechanism. But to carefully rule out large misalignment risks. Some amount of human understanding remains critical.