Jack Fitzgerald

Jack Fitzgerald

@jackfitzgerald.bsky.social

Postdoctoral Scholar at Stanford University. Working on applied econometrics, replication, and metascience. https://jack-fitzgerald.github.io. Likes/reposts aren’t endorsements, views are my own.

1.5K followers269 following195 posts

Jack Fitzgerald
@jackfitzgerald.bsky.social

The article we commented on in PNAS (along w/ many others), which claimed that cognitive scores could be used to detect professional soccer players with 97% accuracy, has now been retracted. Comment: doi.org/10.1073/pnas... Retraction notice: doi.org/10.1073/pnas...

doi.orgRevisiting the cognitive advantages of professional soccer players | PNASRevisiting the cognitive advantages of professional soccer players
Institute for Replication@i4replication.bsky.social

Two more retractions and a discussion of what we learned. 🧵 #CRAVEFiles

Jack Fitzgerald reposted

Institute for Replication
@i4replication.bsky.social

A researcher whose work we were examining asked our chair to investigate concerns about somebody else’s publications at the same university (different field). Our investigation was now generating further allegations from someone already under scrutiny. #CRAVEFiles

Jack Fitzgerald
@jackfitzgerald.bsky.social

My contribution stopped after the first paper, but I can confirm I saw a lot of crazy stuff on at least that first paper. Read our comment at @pnas.org! doi.org/10.1073/pnas...

Institute for Replication@i4replication.bsky.social

Our chair and a few replicators investigated a paper about elite footballers. And missed the bigger story. An independent researcher showed us what we had failed to see. This is I4R’s CRAVE–football investigation. It starts with a PNAS paper. #CRAVEFiles

Jack Fitzgerald
@jackfitzgerald.bsky.social

Yesterday, I had the honor of defending my dissertation and earning my PhD in Economics and Business Administration with cum laude distinction. In October, I will be joining the Meta-Research Innovation Center at Stanford as a Postdoctoral Scholar. I'm excited for the road ahead!

Jack Fitzgerald reposted

Social Science Prediction Platform (SSPP)
@socscipredict.bsky.social

📣 Does involving journal editors in replication material requests impact the availability of such materials? @jackfitzgerald.bsky.social and coauthors invite your predictions! 🕐Time: 15 min 🗓️ Closes: Sept 15 📚 Field: Economics socialscienceprediction.org/predict/r/30...

socialscienceprediction.orgThe Impact of Journal Involvement on Availability of Replication MaterialsI subject the models defending 135 null claims in 81 articles recently published in top economics journals to equivalence testing, which assesses whether an estimate is significantly bounded within a ...
Jack Fitzgerald
@jackfitzgerald.bsky.social

An update on our log-like specifications paper! We’ve updated our recommendations and added new commands in Stata and R for one of our recommended alternatives. 1/ osf.io/preprints/me...

osf.ioOSF
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

IMPORTANT CAVEAT: In quasi-experiments, quantile regression requires extremely strong identification assumptions that are unlikely to hold in practice. We don’t recommend using our commands for that. But they may be useful to analyze RCTs or simple correlations/associations. 9/

Jack Fitzgerald
@jackfitzgerald.bsky.social

An update on our log-like specifications paper! We’ve updated our recommendations and added new commands in Stata and R for one of our recommended alternatives. 1/ osf.io/preprints/me...

osf.ioOSF
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

You might not be able to find a Ψ large enough to meet that criterion, in which case your data doesn’t permit credible estimation using this method (🤷‍♂️). Our commands automate the search for such a Ψ, report results if it can be found, and suppress results if it cannot. 7/

Jack Fitzgerald
@jackfitzgerald.bsky.social

An update on our log-like specifications paper! We’ve updated our recommendations and added new commands in Stata and R for one of our recommended alternatives. 1/ osf.io/preprints/me...

osf.ioOSF
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

Generally, one can run quantile regression after transforming Y with the ‘calibrated extensive margin’ transformation (Chen & @jondr44.bsky.social 2024), setting positive values to ln(Y) and setting non-positive values to constant -Ψ. 5/ doi.org/10.1093/qje/...

doi.orgLogs with Zeros? Some Problems and Solutions*Abstract. When studying an outcome Y that is weakly positive but can equal zero (e.g., earnings), researchers frequently estimate an average treatment effe
Jack Fitzgerald
@jackfitzgerald.bsky.social

If you can find a Ψ* high enough that all median regression-predicted values are < -Ψ*, then the resulting coefficients are interpretable as logarithmic intensive-margin slopes between your covariates and Y’s median, and are identical for any Ψ >= Ψ*. 6/

Jack Fitzgerald
@jackfitzgerald.bsky.social

An update on our log-like specifications paper! We’ve updated our recommendations and added new commands in Stata and R for one of our recommended alternatives. 1/ osf.io/preprints/me...

osf.ioOSF
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

Idea: in mean regression, the point estimate is computed with every value in the outcome, so even one non-positive value messes up logarithmic slope estimation. But quantile regression doesn't use every value to compute the point estimate, just the conditional quantiles. 3/

Jack Fitzgerald
@jackfitzgerald.bsky.social

E.g., say you have a binary treatment. If the median of Y is positive in both treatment and control groups, then even if there are some non-positive values in the data, the logarithmic difference between those medians is defined, and can be recovered using median regression. 4/

Jack Fitzgerald
@jackfitzgerald.bsky.social

An update on our log-like specifications paper! We’ve updated our recommendations and added new commands in Stata and R for one of our recommended alternatives. 1/ osf.io/preprints/me...

osf.ioOSF
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

Jack Fitzgerald
@jackfitzgerald.bsky.social

Happy to have the CBS Replication Games (and my replication research more broadly) featured on the ODISSEI Podcast! Tune in if you're interested in hearing @drtomemery.bsky.social and I nerd out over replication!

ODISSEI@odissei.bsky.social

How can games change research culture? 🧩 On this month's ODISSEI podcast, @drtomemery.bsky.social talks with Jack Fitzgerald about replication, and why it needs to be more embedded in research. That's why the Replication Games were developed, Jack explains. 🎧 Tune in to learn more: edu.nl/eqhuc

Jack Fitzgerald reposted

ODISSEI
@odissei.bsky.social

How can games change research culture? 🧩 On this month's ODISSEI podcast, @drtomemery.bsky.social talks with Jack Fitzgerald about replication, and why it needs to be more embedded in research. That's why the Replication Games were developed, Jack explains. 🎧 Tune in to learn more: edu.nl/eqhuc

Jack Fitzgerald
@jackfitzgerald.bsky.social

Our paper on the effect of large language models on replicators' success in conducting reproducibility + robustness checks is now published at PNAS. Before advocating that we let AI completely take over the replication process, please read our paper! doi.org/10.1073/pnas...

doi.orgPNASProceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans...
Institute for Replication@i4replication.bsky.social

9/ The paper is "AI-Assisted Teams Outperform AI-Led Teams but Not Human-Only Teams in Assessing Research Reproducibility in Quantitative Social Science." Published in PNAS. DOI: doi.org/10.1073/pnas... Preprint: ideas.repec.org/p/zbw/i4rdps...

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

So most unrepresentativeness in business survey comes down to which businesses are contacted. That means there's a lot of potential to improve representativeness, generalizability, and/or response rates by understanding and modifying the design of business surveys. 11/

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

Most worrying: 70% of establishments in the LISA register are registered at a residential address. These establishments are 18 p.p. less likely to respond to LISA's business surveys than the average office, holding one of the lowest response rates among facility types. 9/

Jack Fitzgerald
@jackfitzgerald.bsky.social

But controlling for contact probability erases most differences in business survey response, reducing maximum response rate differences between sectors from 32 to 9 p.p. Establishments registered to residential addresses get response rate deficits cut from 18 to 5 p.p.! 10/

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

Responsive establishments hire fewer people, who are less likely to work fulltime. This is consistent with efficient division of labor: the sorts of firms efficient enough to delegate fulltime and parttime tasks are more able to allocate administrative labor to answer surveys. 7/

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

Critically, conditional on a number of stratification/exception variables, LISA's business surveys are randomly distributed. So if you control for those variables, you can isolate which firms are most likely to respond to business surveys *conditional on being contacted*. 5/

Jack Fitzgerald
@jackfitzgerald.bsky.social

About 19% of Dutch establishments provided responses to LISA’s surveys in 2022. Nearly all of the remaining data is imputed from prior years. In my paper, I discuss LISA's imputation methods and propose alternatives, which I show empirically improve predictive performance. 6/

View full thread
Jack Fitzgerald
@jackfitzgerald.bsky.social

But in the Netherlands, there is a 'business census'. Each year, the Landelijk Informatiesysteem voor Arbeidsplaatsen (LISA) runs regional business surveys and combines data from these surveys with administrative data on all establishments in the Netherlands. 3/

Jack Fitzgerald
@jackfitzgerald.bsky.social

This LISA data can be provided to researchers at certain Dutch institutions through the FIRMBACKBONE infrastructure, and I leverage this data to examine the differences between establishments that do and don’t respond to LISA’s regional business surveys. 4/

Jack Fitzgerald
@jackfitzgerald.bsky.social

Like all surveys, business surveys have the problem that respondents often aren't representative of the population. But in human surveys, you can deal with this by comparing your sample to, e.g., a census. Problem: in most settings, there's no 'business census' to compare to. 2/

Jack Fitzgerald reposted

Peder M Isager
@isager.bsky.social

My article "Three-Sided Testing to Establish Practical Significance" with @jackfitzgerald.bsky.social is now published in AMPPS! journals.sagepub.com/doi/10.1177/... Three-sided testing is an improved version of TOST that lets you test for equivalence, superiority and inferiority simultaneously.

journals.sagepub.comSage Journals: Discover world-class researchSubscription and open access journals from Sage, the world's leading independent academic publisher.
Jack Fitzgerald
@jackfitzgerald.bsky.social

Happy to see my paper with @isager.bsky.social on three-sided testing published at Advances in Methods and Practices in Psychological Science! If you're interested in practical significance testing, give it a read! doi.org/10.1177/2515...

doi.orgThree-Sided Testing to Establish Practical Significance: A Tutorial - Peder Mortvedt Isager, Jack Fitzgerald, 2026Researchers may want to know whether an observed statistical relationship is either meaningfully negative, meaningfully positive, or small enough to be consider...
Jack Fitzgerald@jackfitzgerald.bsky.social

New paper for holiday reading! @isager.bsky.social and I provide an introduction to three-sided testing, a framework for testing estimates' practical significance. We offer a tutorial, Shiny app, + commands/code in #Rstats, #Jamovi, + #Stata. 1/9 osf.io/preprints/psyarxiv/8y925 #EconSky #PsychSky

Jack Fitzgerald
@jackfitzgerald.bsky.social

Given my stance on log-like specifications, I was surprised to learn that there's a 'news' article on my paper in Nature Human Behaviour, claiming that it actually advocates for the use of IHS specifications. This is categorically untrue. 1/ t.co/ZGjAmHZhLv

t.cohttps://scienmag.com/imputations-and-inverse-hyperbolic-sines-unveiling-data-mysteries/
Jack Fitzgerald@jackfitzgerald.bsky.social

New preprint! We reanalyze 46 papers that use log-like specifications (ln(Z+1), inverse hyperbolic sine etc). We find widespread non-robustness, and we show through theory + simulation how these models drive spurious significance. 1/ doi.org/10.31222/osf...

Jack Fitzgerald
@jackfitzgerald.bsky.social

My NHB paper is literally about an article where problems with IHS transformations revealed data irregularities that (partly) resulted in that article's retraction. That NHB paper doesn't take any stance on IHS specifications, let alone a *pro* stance. 2/ doi.org/10.1038/s415...

doi.orgImputations, inverse hyperbolic sines and impossible values - Nature Human BehaviourNature Human Behaviour - Imputations, inverse hyperbolic sines and impossible values