Post

They trained an AI model on a widely used knee osteoarthritis dataset to see if it would be able to make nonsensical predictions - whether the patient ate refried beans, or drank beer. It did, in part by somehow figuring out where the x-ray was taken. www.dartmouth-health.org/about/news/a...

The authors point out that AI models base their predictions on sneaky shortcut effects all the time; they're just easier to identify when the conclusions (beer drinking) are clearly spurious. www.nature.com/articles/s41...

Algorithmic shortcutting is tough to avoid. Sometimes it's based on something easy to identify - like rulers in images of skin cancer, or sicker patients getting their chest x-rays while lying down. But as they found here, often it's a subtle mix of non-obvious correlations.

AI models are not approaching the problem like a human scientist would - they'll latch onto all sorts of unintended information in an effort to make their predictions. This is one reason AI models often end up amplifying the racism and gender discrimination in their training data.

Just finished the book. While it was launched before GPT popularity, it’s amazing how relevant it still is. Got confess: I laughed out loud while reading about the “ruler-detector”. And I’m really enjoying your blog!

Thanks so much! It’s amazing how many ruler detectors still get built!