Post

"seeing" something that isn't there is literally the definition of "hallucination" so i think these researchers had to know what they were doing & were doing it on purpose

the models generally "championed" for radiology are typically specialized, not commercial chatbots aka "frontier models" they still do the stupidest shit — like correlating diagnosis based on which hospital name was on the image — bc there's no such thing as artificial *intelligence*

answer-shaped extrusions can be useful in some situations but literally zero of those situations are where you need 1. facts about reality 2. truth 3. probability 4. certainty 5. important decision-making

I saw something similar a couple years ago about AI more accurately diagnosing TB than doctors. Turns out it was just dating the image, and older images in the dataset were more likely to be of a TB patient.

The LLM "cheats" on the benchmark; giving the right answers despite having not the questions in full shows that it cheats by reproducing an answer rather than doing any "analysis". This may be obvious to you, but it is not obvious from the claims made about "potential" let alone "intelligence".

it can’t “cheat” because it is not taking a test or giving answers. it is simply playing improv.

also i have no clue what you mean with that “it may be obvious to you” jab. you think *researchers* shouldn’t know this? because wow that’s really something