Post

Grace
@gracekind.net

There’s some cool recent research on this phenomenon! It turns out vision language models excel at image benchmarks *even when the actual images aren’t provided,* because the answers are implicit in the questions! arxiv.org/abs/2603.21687

arxiv.orgMIRAGE: The Illusion of Visual UnderstandingMultimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We r...
Kyle L@haikugoat.bsky.social

It's really instructive to look at an LLM's output and piece by piece look to see how much of it was already implicit in the prompt, and where exactly. It makes me realize how much I'm telling on myself all the time.

affine
@refinement.systems

"(...) by generating a reasoning trace, indistinguishable from a correct one, solely based on a described mirage rather than a real input, while at the same time scoring high on the benchmarks. Indeed, these models outperform not only other AI models, but in some cases also human experts." wtf

0 reposts7 likes

No replies yet.