Post

It's surely just regurgitating from the many counterexamples to the Jacobian Conjecture that were in the training data

Ethan Mollick@emollick.bsky.social

Well, the Jacobean Conjecture appears to have just been proven false by Fable.

14 reposts173 likes

the funny thing is that if this were true, it would actually definitively prove that LLMs are a new type of useful search engines

Everyone is just taking that tweet at face value? Assuming that result is correct, we don't currently have verification it was done just recently - vs. a project they secretly devoted months to then chose to reveal at the most viral moment. Meanwhile, plenty of outside proof for both positions.

computerworld.comAI chatbots are wrong about news 45% of the time, study findsGoogle's Gemini was the worst performer; it had problems with 76% of its responses, according to an international study of public service broadcasters.
Eerke Boiten@eerkeboiten.bsky.social

And the top rating for AI cheating goes to: "In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images." futurism.com/artificial-i...

sorry but "they actually had this outcome a while ago and sat on it" is a really silly conspiracy theory. I cannot think of an upside and your links are totally unrelated? It's not a surprise that LLMs are bad at news, especially without context? imo very much a misuse of the technology

I don't think this is what happened here, but I could imagine a company throwing a lot of compute at math problems, sifting out the good results, and then drip feeding them out to keep enthusiasm up and balance bad PR. Seems like a pretty sensible strategy to me, actually.