JK Flake

And this is why meta analysis is screwed

Sorry I gave them so much hell on this one but happy to see some solid discussion about the necessity of qualitative approaches. Bravo.

Our lab has three RRs of simulations underway. It’s a shame to tell my students that our top journal will not consider them for publication. Let’s change that!

Help us show there is support for offering registered reports at one of Psychology's leading method's journal! chng.it/TwwnVBScVb

"Preregistration" lumps three distinct practices into one: - Study registration - Protocol - Analysis plans (SAPs) To advance open science, it's critical that we distinguish them. 🌟 🌟 Now in-press paper from Evan Mayo-Wilson, @seangrant.bsky.social, David Moher, me: osf.io/preprints/me...

AMPPS Call for Papers: Replicability and Reproducibility in Methodological Research. Proposals due September 15. @jkflake.bsky.social 

Folks, it's time to submit your papers to PCI Psychology. Here is our invitation to you, taken from our editorial osf.io/preprints/ps...

Check out this awesome new paper about #OpenScience in developing countries! Paper is forthcoming at AMPPS- very proud that we can promote this work. Congrats to the large and international authorship team, and esp @hcp4715.bsky.social for leading the charge. @psychscience.bsky.social

A new preregistration template for simulation studies is now live on OSF. Thanks to the creators and the preregistration template working group for proposing and shepherding this to make it available to users. See the post if you would like to propose a new template. www.cos.io/blog/introdu...

Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.

🚨 Calling all dog behavior & cognition researchers - ManyDogs Project is launching ManyDogs 2! 🔍🐕‍🦺We're studying overimitation: Do dogs copy irrelevant actions just b/c their favorite human does them? Email manydogsproject2@gmail.com to collaborate globally and contribute to reproducible science!

I'm very happy to see meta-science on methods research getting more attention in the form of a great new call for papers! I recently gave some talks on the existing literature on the quality and transparency of sim studies & ideas for future improvement. Slides: bsiepe.github.io/talks/2025-0...

Are you working on issues of replication and reproducibility of methodological research in psychology? Consider submitting a proposal to my call for papers, happy to chat about your ideas or answer any questions! www.psychologicalscience.org/publications...

🎉 After 2 years of work, our #OpenScience guide for researchers in developing countries is LIVE! Check out our new preprint: osf.io/preprints/ps... We map the opportunities 🌟 + barriers 🧱 facing Global South researchers. *(1/4)*

PCI Psychology is here!! 🎉🥳 After over a year of hard work by so many people, we are thrilled to announce that we are open for submissions! Join us in making publishing more efficient, equitable, and open: psych.peercommunityin.org #PsychSciSky #scipub

Robustness analysis is going to be a big part of the future of how we think about evidence synthesis and rigor - exciting new journal where you can turn your blog posts of other ways the analyses could have been done into papers! Excited to see some submissions come in!

In the workshop we'll follow an example of developing a survey to measuring pickle fanatism for marketing purposes to learn about how to develop and evaluate survey questions for a variety of purposes. The workshop spans qualitative and quantitative methods. Two days of measurement fun!

Chuffed and excited to announce a new journal section in Cortex: *Methods and Assumptions* is open for submissions starting today! Read the opening editorial here (OA): www.sciencedirect.com/science/arti... 1/

Attention metascience & AI enthusiasts: SSHRC is offering $140K/2yr postdoc fellowships, and I'd be happy to sponsor someone to work in my lab at UofT. If you're interested in how AI might transform scientific practice (for better or worse), apply by April 8. Let's question everything together.

I have to fix it in my head. This survey validates me.

I’ve long used FiveThirtyEight’s interactive “Hack Your Way To Scientific Glory” to illustrate the idea of p-hacking when I teach statistics. But ABC/Disney killed the site earlier this month :( So I made my own with #rstats and Observable and #QuartoPub ! stats.andrewheiss.com/hack-your-way/

Hope to see more papers like this that question the performance and trade-offs of more complicated and newer methods with a sample of datasets (not just simulations that can benefit from cherry picking). Combats the publication bias towards new methods and demonstrates on-going comparative work.

I'm hiring! I'm looking for 1️⃣ pre-doc and 2️⃣ post-docs. If you're fascinated by media effects, quantitative methods, and open science, please consider applying 🙂 International applications welcome! Application Pre-doc: jobs.uzh.ch/job-vacancie... Application Post-doc: jobs.uzh.ch/job-vacancie...

👀

HelioCampus is hiring! We have openings for data scientists and data analysts. www.heliocampus.com/company/care... I'm the hiring manager for the data scientist role, and we're specifically looking for someone with experience working with student data in the higher ed industry.

The methods and statistics group at @tilburg-university.bsky.social get it.

The looks great: “cross-cultural developmental research poses distinct problems for ensuring high construct validity, owing to the nuances of working with children; the standard approach of transporting protocols designed & validated in one population to another risks low construct validity”

😱 'The sheer number of EFA structures that was deemed 'passing' by CFA fit indices made me realize that perfect simple structure is really more of an assumption than a real tested proposition in CFA.'

Where is the hand brake? Hand brake hand schmake @eikofried.bsky.social

Learning about big team science projects with @debruine.bsky.social on day 3 at @psicostat.bsky.social winter school. Good chat about less sexy aspects of working in diverse groups: differences in ethics approval and date formats. Put the year first! 2025-02-19

We submitted to Frontiers?! 😱 It's a comment on a Frontiers paper (doi.org/10.3389/fpsy...) purporting to measure “problem gambling.” But they didn't! The scale was edited: “gambling” was changed to “spending money on gacha.” So it no longer measured “problem gambling.” 🤦 doi.org/10.31234/osf...

I didn’t realize I needed a thread with conference highlights, much better than an email or remembering to check a website. So excited to see everyone in Minnesota this summer!

Visual Analogue Scales are often used to measure short-term changes in body dissatisfaction. But how much change (out of 101 points) represents a meaningful change? 20 points? 10? 5? In our new (open access) paper, we try to find out: www.sciencedirect.com/science/arti...

Re sabbaticals: how do you handle your lab/on-going research? Do you cut out or scale back? Do you work less or does research just fill the time that was teaching/service? If you travel/do research visits/try to learn things, how do you also run your lab? Do you tell people you'll be slower?

I just want to say when I called my lab, "flake lab" it was because I couldn't think of a name for it, and well it is my lab. But now it is routinely referred to as "flake lab" -- I'm sorry for all of those involved. Do we study flakes? No, we do not.

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n