Lisa DeBruine

Lisa DeBruine

@debruine.bsky.social

Professor of MetaScience 🏳️‍🌈 she/they Bacon-Erdös-Sabbath number: 10 #metacheck #rstats #PsyTeachR #OpenResearch #ManyFaces (overwhelmed by social media)

7.2K followers426 following749 posts

Lisa DeBruine reposted

Replying to @mattansb.msbstats.info

You know how the "No brown M&M's" clause in Van Halen's contracts works? (summary: https://en.wikipedia.org/wiki/Van_Halen_test) Well that's what demanding a simulation-based power analysis does for a complicated, e.g. multilevel model: forces you to understand that model properly before showtime.

Lisa DeBruine reposted

I needed to know what day the semester ends at my own university; it only took me a web search and 8 separate clicks to find

Lisa DeBruine reposted

Metacheck is now hosted by Surf! This offers us the possibility to offer an all in one online solution. Before we advertise this more, we would love to trial the new tool with data managers and editors of Diamond open access journals who want to try out automated checks. Especially of data and code.

Lisa DeBruine reposted

WTF is wrong with autocorrect lately? I just misspelled “maintenance” as “maintainance” and all it could suggest was “maintain ancestors” and “maintain anchovies”

Do I have any co-authors with an Erdös number less than 3? I just found out my Bacon-Erdös-Sabbath number is at most 10, so checking if I can get it any lower.

Yes! Power should be a property of a single hypothesis test, even if that consists of multiple analytic tests. 1 hypothesis = 1 test is just the simplest case, but you can calculate power by simulation for arbitrarily complex analyses and decision rules.

Suzanne Segerstrom@segerstrom.bsky.social

Statistical power is a property of an analysis, not a study. Thank you for coming to my TED talk.

View full thread

Simulate data from a null distribution (A:D all same) and re-run the simulation to calculate the false positive rate for your set of tests. This should be the same as your alpha criterion for single tests, but gets more complex with multiple dependent tests.

Don’t @ me about using contrasts being better here — it’s just an example. But also sometimes I think we overcomplicate analysis trying to turn a complex decision rule into a single number because we don’t have frameworks and tools to think about power and false positives across multiple tests.

Yes! Power should be a property of a single hypothesis test, even if that consists of multiple analytic tests. 1 hypothesis = 1 test is just the simplest case, but you can calculate power by simulation for arbitrarily complex analyses and decision rules.

Suzanne Segerstrom@segerstrom.bsky.social

Statistical power is a property of an analysis, not a study. Thank you for coming to my TED talk.

To elaborate: if your decision rule for supporting an H is that condition A > B:D and that the differences among B:D are significantly smaller than an SESOI, you can simulate this, run all analyses, and decide whether to reject the H. Do this many times and the proportion of non-rejections is power.

"I work with collaborators whose opinions and values are important to me, and several of them are more skeptical about the use of AI." I think I might be one of those collaborators :)

Daniel Lakens@lakens.bsky.social

In preparation for a discussion with the lab on AI use tomorrow morning, I thought I would come prepared and write down my thoughts in a blog post: How I use AI in my work daniellakens.blogspot.com/2026/10/how-...

View full thread

@lakens.bsky.social has changed my mind in the past (like returning to vegetarianism) and I can see more ways now that AI might fit into my workflow. Although I'll probably always lean more towards wanting to understand the intricacies of code so I can wield that knowledge for things AI can't do.

"I work with collaborators whose opinions and values are important to me, and several of them are more skeptical about the use of AI." I think I might be one of those collaborators :)

Daniel Lakens@lakens.bsky.social

In preparation for a discussion with the lab on AI use tomorrow morning, I thought I would come prepared and write down my thoughts in a blog post: How I use AI in my work daniellakens.blogspot.com/2026/10/how-...

View full thread

I really like this blog post. The use for coding conforms to what I tell my data skills students: you can use any tool you want, including AI, only after you thoroughly understand what your code is meant to produce. Years of work with data and code have given @lakens.bsky.social this understanding.

I agree 100% with the opinions on AI for writing. "If people use AI to write, maybe what they are writing does not need to exist." (Although perhaps AI should have been used to fix this sentence: "As enthusiastic I am about using AI for code, as disappointing I am in using AI to write.")

"I work with collaborators whose opinions and values are important to me, and several of them are more skeptical about the use of AI." I think I might be one of those collaborators :)

Daniel Lakens@lakens.bsky.social

In preparation for a discussion with the lab on AI use tomorrow morning, I thought I would come prepared and write down my thoughts in a blog post: How I use AI in my work daniellakens.blogspot.com/2026/10/how-...

For context, I listened to this blog post using text-to-speech while knitting my first stranded colourwork, something a machine could do better than me, but learning to do gives me skills and insights I can use for the things a machine can't do.

This post does a great job of articulating why the development and release of metacheck has been pretty slow compared to other tools. One of our core values is validation, which is one of the things that has not been made a lot faster by LLMs. www.scienceverse.org/metacheck/#o...

Ian Hussey@ianhussey.mmmdata.io

Our reflections on the recent "AI and Research Integrity" meeting in Montreal: We are somewhat worried that many AI trustworthiness assessment tools are not trustworthy. @lhdjung.bsky.social @jamiecummins.bsky.social trustworthy.scientific.claims/posts/trustw...

We should be making some development announcements soon, but want to be sure we have the validation info to back it up. And we want this supporting material to be archived in as high quality as what we are recommending with the tool. (Some very cool repo- and data-checking modules are coming :)

Lisa DeBruine reposted

any statistician born after 1983 can't hand-calculate sums of squares... all they know is be Bayesian, eat hot chip & calculate marginal effects

Lisa DeBruine reposted

Replying to @tressiemcphd.bsky.social

So you know previous waves of technological fabulism the tech companies threw a little loose change at social behavioral research and user interface experience. whats interesting about AI interesting is that they aren’t even pretending to care about how people use the technology.

Lisa DeBruine reposted

Four days ago, a scientist was surprised by the news that Anthropic discovered new virus genes for making DNA. He says he’s been studying them for years—and feeding his data to Anthropic’s AI as part of his own research. Coincidence? Here’s my story. nyti.ms/3VSWayc

nyti.msDid Anthropic’s A.I. Really Make a Scientific Discovery on Its Own? (Gift Article)An expert at the University of Copenhagen said his team had been sharing its research with the company’s A.I. model, Claude, and that its new finding matched their work.

I love that you can see now if people follow you from a starter pack. And I love even more the fact that most of my new follows come from one called "Science Shitposters".

Lisa DeBruine reposted

Survey of German academics finds restrictions on academic freedom are rare and not associated with academic position, discipline or gender, but expectations of restrictions are higher among precarious academics. Hope this paper gets attention, as it does not confirm media stereotypes of academia

Julian Hamann@julianhamann.bsky.social

Happy to share our new article on academic freedom. @paulita.bsky.social, Christiane Thompson and Gregor Fabian examine how academic rank shapes experiences, expectations and perceptions of restrictions on academic freedom, based on a survey of 9.083 academics in Germany. doi.org/10.1111/1468...

Lisa DeBruine reposted

Journals should stop offering to store supplemental information. Everything that is shared with a paper should have it's own DOI, and live in a proper repository. As authors, link to files (e.g., Zenodo), but never add them as supplemental files.

Lisa DeBruine reposted

Machine learning and generative LLMs are subject to many of the same drawbacks and issues as general empirical research in the typical publishing system. In this piece now published at NHB we argue that, like for empirical research, ML and LLMs can benefit from the use of Registered Reports.

Lisa DeBruine reposted

Workshop done 😀 since I anyway put together a list of resources, I'm going to share it in this thread. So here we go, my top recommendations for how to improve computational reproducibility of your work

Ana Martinovici@anamartinovici.bsky.social

I am giving a workshop on "good (enough) practices for computational reproducibility" tomorrow. Can't wait to share how much fun it is to use Git

Lisa DeBruine reposted

`webexercises` creates interactive exercises for pkgdown, Rmarkdown, and with this update, Quarto (books), Latex output, and #R package vignettes! My hope is that this will support open educational materials, especially in R-package documentation. cran.r-project.org/web/packages...

cran.r-project.orgwebexercises: Create Interactive Web ExercisesFunctions for easily creating interactive web exercises in 'R Markdown', 'Quarto', and package vignettes that students can use in self-guided learning.

I was pleased when most of my cool academic colleagues got on here and there was more interesting content, but lately every time I log on, I end up depressed about the world and contemplating just giving up and joining a commune with my extended polycule.

Lisa DeBruine reposted

1/ What is the prevalence of personally identifiable information (PII) in author-collected data publicly shared in the social sciences? We audited 327 replication packages from 11 leading journals in economics, political science, and psychology Answer: surprisingly high 🧵

I love the concrete examples of how to avoid anthropomorphism in this article: artificial intelligence → probabilistic automation image recognition → image labeling speech recognition → automatic transcription shows bias → reflects bias hallucination → undesirable output goal → success condition

Emily M. Bender@emilymbender.bsky.social

A good rule of thumb: If the press release (or stenographic news reporting masquerading as journalism) says that an "AI" did something, that's deliberate obfuscation. Software doesn't do anything of its own accord, but folks sure like to talk about it as if it does: >>

We have a similar problem in evolutionary sciences. It’s really hard to talk about a non-conscious process without anthropomorphising it. Among experts, it’s metaphorical and shorthand, but too likely to mislead non-experts.

Lisa DeBruine reposted

This is, incidentally, ALWAYS where the anti-trans playbook was going to lead, because they fundamentally conceptualized being trans as having a mental disability, and every last scrap of ableism they were covert with there, they're mask off on now. They must be rejected--in whole. Hobbes is right.

Michael Hobbes@michaelhobbes.bsky.social

However much you hate British media, it is not enough www.telegraph.co.uk/news/2026/09...

Lisa DeBruine reposted

If you know the details of the Claudine Gay (Harvard) and Jason Arday (Cambridge) controversies but little about the Marc Tessier-Lavigne (Stanford) controversy, you should ask yourself why. [And, yes, the answer is as simple as you think it is. This isn’t 3-dimensional chess...]

I got small town southern Ontario, which I guess is pretty accurate as I lived in Hamilton, ON for three years.

ℳatt@matttomic.bsky.social

I've seen everyone sharing that US Dialect Quiz so I decided to make my own, much crappier, CANADIAN DIALECT QUIZ to guess where in Canada you're from

I could support a requirement that your co-authorship team will provide the same number of reviews your paper requires (presumably 2) over the next year, and that if you don’t do this you need to explain in the cover letter of your next submission (with a policy to only punish egregious abuses)

This post is unavailable.
View full thread

I would really worry that this sort of policy would encourage AI-generated reviews or very low quality reviews by people with insufficient expertise, just to fill the requirement.

I did the distributed reviewing for the MetaScience ESRC grant call and found it reasonable and educational, but also had a team to divide the 7 reviews among (we were given 8 but had one conflict). This policy would absolutely make it way more difficult for a single applicant.

I could support a requirement that your co-authorship team will provide the same number of reviews your paper requires (presumably 2) over the next year, and that if you don’t do this you need to explain in the cover letter of your next submission (with a policy to only punish egregious abuses)

This post is unavailable.

But then I guess the argument is that the people most likely to be taking advantage of the system will also be likely to lie about the reasons they couldn’t review?