Post

One more set of meta comments on this: after OpenAI's announcement of Navier-Stokes, they got lots of pushback from the math community. One of the things OAI did in response was to ask for recommendations from an independent advisory board of mathematicians on how to conduct future releases.

This group is called AGMAI (Advisory Group on Mathematics and Artificial Intelligence). In late September, they released their set of recommendations: agmai.org/general-sep29/. Let's go over some of the recommendations and see which ones OpenAI followed vs. the ones they did not.

First, they outline two possible scenarios: one where a human understands the AI proof; essentially, this is where the AI is copilot, and how many mathematicians do research today. They advise no change from how mathematics has been distributed in the past.

The other case is when no human understand the AI proof (in all its detail). This is the case for essentially all recent OpenAI proofs. Here they have several detailed suggestions:

1a: "The literature should be scoured for any ideas that are related to the ideas in the proofs of the results released. Even if the AI lab’s model discovered those ideas independently, it should follow standard mathematical practice and cite the papers in which the ideas were first introduced."

Obviously, it is impossible to audit this within a day of release. But this was one of the main complaints of the NS paper, to which additional citations were silently added a few days after the first release. So if that is any precedent, it is unlikely recommendation was systematically followed.

1b) "The model, or some other model, should be prompted to produce a version of each proof that is written up in a style that follows the conventions of a traditional mathematical paper..." Again, just based on a skim, this set of papers seems to do reasonably well here.

They have introductions, theorem statements, outlines of key ideas in the proof, etc. Of course, LLMs excel at this sort of summarization. However, I do not know if the details of the technical exposition are as friendly to read. LLMs are notorious for producing lots of dense jargon.

"3. For each result released, the AI lab should make public the name of the model, the prompts used, a (summarized) chain of thought, the time taken, and the estimated cost of computation" ❌ This did not happen. The model was internal (it doesn't have a public name!), ...

Hardly any prompts or chain of thought was revealed, and no precise token counts were given (only an averaged estimate of time spent per problem).

"4. As far as possible, a proof released by an AI lab should be formalized." ✅/❌: Partial credit here! Some proofs were formalized on release, but many were not. Apparently this is work in progress? It is unclear why they were not all formalized.