Simon P. Couch on Mythryl: “One of the more interesting findings to me from this eval is that…”

One of the more interesting findings to me from this eval is that prematurely adding modeled results (like #rstats geom_smooth()) to plots drastically reduces the chances that models will catch the issue in the plot.

Simon P. Couch@simonpcouch.com

We're excited to share bluffbench2, a next-generation data science LLM evaluation. The eval measures how well models notice subtle data quality issues in plots. Even the most capable models today score in the mid-teens. opensource.posit.co/blog/2026-07...

opensource.posit.coAI Newsletter: LLMs often miss subtle visual artifacts in data visualizationsIntroducing bluffbench2, a new data science LLM evaluation.

21/7/26, 1:38 pm0 reposts7 likes

Post