Steve Haroz

BLOG POST: Invalid Conclusions Built on Statistical errors If you see a small p-value or a large separation in confidence intervals, you may assume that an effect is reliable. But overly simple reporting may be hiding serious conclusion-altering errors. Here are some example scenarios: 1/🧵 #stats

Your monthly reminder that if you're using ChatGPT to do your science, you're failing your colleagues, community, and all of us. Here, ChatGPT makes up data and presents it as the real thing — then apologizes when confronted.

Genuinely, is this it for science communication? Just hopelessly fighting off a torrent of stupid bullshit spewed out by genAI that people mindlessly repeat?

The purpose of peer review is not primarily to help authors improve their papers.

Is Ultra-processed food destroying our brains? Despite the recent headlines probably not. My new piece. gidmk.substack.com/p/is-ultra-p...

Provocative new preprint examines 14 published meta-analyses claiming AI improves student learning and concludes... Not a single one provides a valid basis for its claims, citing widespread methodological flaws. osf.io/preprints/ed... Watch a webinar from yesterday about the findings:

I really need ton convert my (php-based) wordpress blog to something static. Sigh... yet another project added to the todo list.

What’s the relationship between different aspects of visual time perception? To address this question, I propose a classification in four clusters of twelve classic visual time phenomena. Comments/criticisms welcome, including by email. osf.io/preprints/ps...

We analysed over a thousand academic job ads in Norway from 2023 and 2025. Very few mentioned open science practices as desired or required skills, and this didn't change over time. However, we *did* find that open science was mentioned more often in ads for tenured positions than early career ones

We were investigating Wedgewood for faking their data and emailed them for comment this morning, ahead of publishing the findings tomorrow. They are now trying to cover up their fraud with the "we were doing a social experiment" excuse. Full story here: blog.fiftyplusone.news/p/fiftypluso...

Convert a Qualtrics survey description into a PDF report showing all the branches, questions, and answers options. Hugely convenient for appending to a paper. (From @datacolada.bsky.social) datacolada.org/140

Double and triple negatives should be forbidden on voter initiatives. "Are you in favor or against reversing the repeal of the removal of the tax?"

Yeah, my point was that if you're going to guess what seed a random number generator used, 42 is a pretty good starting point. My point was more to dismiss the idea that the sleuth needed insider knowledge to figure a very commonly used seed.

New post! In which I discuss a truly amazing piece of science sleuthing that recently appeared on @pubpeer.com neuroskepticbsky.substack.com/p/the-sherlo...

I've always felt that #rstats's ranking in the TIOBE index felt a lot like random noise, but every time it gains or drops significantly TIOBE tells a little story about it. So I extracted all those stories into one place: gist.github.com/hadley/1af6d...

Apparently, a VPN server in the Seychelles tried to log into my home server last night. They didn't get in. But I'm amused at the idea of trying to explain a VPN server in the middle of the Indian Ocean as having any legal utility.

Most dashboards are an infodump of statistically meaningless numbers. If people want to automate making them with genAI, then good riddance.

Are the original Johnson et al. 1991 findings evidence of a face-template or something more general, and is this bias to orient towards stimuli like them reflective of an domain-specific mechanism? This excellent paper challenges the face-template interpretation as well as the domain-specific one.

Currently, a diagram is making the rounds that is claimed to show a significant reduction in Russian disinformation bot activity after Ukrainian drones hit Yandex data centers. There is no evidence that this plot shows real data. I suspect that this image is AI-generated.

The problem is noise, not N: why increasing the sample size isn’t all that. statmodeling.stat.columbia.edu/2026/10/10/t...

Color constancy: the circled areas have the same color even if your visual system insists otherwise.

I don't understand how research papers so frequently misreport the sample size. It's literally the easiest number to get right. And every number after it is called into question if it isn't accurate. Anyway, here's an overview of how I start a review: steveharoz.com/blog/2022/re...

Another special issue at Psych Science! We invite “submissions that advance the field through critical appraisal of articles previously published by Psychological Science” Please share! www.psychologicalscience.org/publications...

A really interesting person with an underappreciated impact

Behavioral, cognitive, and social science research has had devastating cuts in the US in the last couple years.

Here's the figure from the supplement with different tasks used for "cross-validation" of the bot. You can see that it often under- or overestimates humans. So sometimes it can be filtered out as a very poor subject, but would you automatically discard a very good one? 4/n

Search engines be like: I'm gonna ignore that first word that you put in quotes and added a + in front of and just return every result that has the second word.

Here’s the monthly data with a 12-month trailing average, so we don’t “look ahead” of the time series. We can see that visits from Canada are substantially down. Like, a *lot*. And if we looked at the French series by itself we’d see something too. But it’s not on the scale of Covid. Yet.

TL;DR Task-relevant grouping helps reduce misinterpretation for graphs journals.sagepub.com/doi/10.1177/...

#TodayinHistory #dataviz #OTD 📊 🎂Sep 29, 1829 Michael George Mulhall born in Dublin, Ireland 🇮🇪 1884: 🥇1st? use of a pictogram, to represent data by icons proportional to a number The train one is accurate; the wheel one commits the area illusion, using diameter to represent the number

So the whole giant story about 5 men being stopped from performing a major terrorist attack on a UK air force base involved in the US war on Iran ... is looking like just some local guys stealing petrol from nearby farm equipment. 🤡

If you can spare ~20 min on an online study on perception & #mentalimagery please consider participating: #visionscience #psychscisky #neuroskyence tstbl.co/131-782

If I search for tropical cyclones that actually came within 100 miles of New Mexico, I found 16 of them going back to 1916. The most recent was Newton in 2016. This isn't just remnants bringing rain, these are actual data points as tropical cyclones.

🚨New paper alert! 🚨 The visual system tunes itself to the statistics of its input. How do individual experiences differently tune vision? In this work, @bjbalas.bsky.social and I examined how one individual difference—height—changes the visual diet and how this affects perception. 1/

💯

I think I have cracked one of the final things that kept people in keynote over quarto revealjs slides. Introducing ✨️ quarto-revealjs-magic-move ✨️ emilhvitfeldt.com/post/quarto-... #quarto #slidecrafting

New preprint! 🎉 I analysed 1660 papers from 4 psychology journals and found materials sharing went from 9% of papers in 2015 to 82% in 2025, and these materials *do* get downloaded — a median of 135 times each. BUT shared code is often hard to run. doi.org/10.31234/osf... Let's walk through it 🧵

Amazing edshee.run

Many people are disappointed with the current state of technology, but the "popcorn" button on modern microwaves does actually work now.

I made it through my #PositConf2026 talk! The Design of Everyday #RStats Functions. Slides here: stbl.wrangle.zone/slides/every... The underlying 📦 {stbl} is on CRAN!