AEA Data Editor

Very useful from @datacolada.bsky.social , can help make your own (or others) replication packages more transparent! Note: if you are submitting to an @aeajournals.bsky.social , you are required to provide your QSF, or LimeSurvey, or SurveyCTO, or .... files. This can help you make them clearer.

Data Colada@datacolada.bsky.social

Data Colada[140] ResearchBox now converts QSF Qualtrics Files to PDF in About 1 Second. datacolada.org/140

PSA: Please be precise when using Github as input to scientific articles OK, this is going to be nerdy... If you use Github as an deposit for your scientific output, or you are re-using somebody else's Github deposit, please be precise. I have some thoughts...

Also helps with that tenure thing, because now you have a DOI that might be counted via citation metrics!

aeadataeditor.github.ioPSA: Please be precise when using Github as input to scientific articles.If you use Github as an deposit for your scientific output, or you are re-using somebody else’s Github deposit, please be precise. I have some thoughts…

Like I said: not legal advice. But hopefully good advice. See aeadataeditor.github.io/posts/2026-0...

Use CITATION.cff! docs.github.com/en/repositor... It, too, is highlighted by Github in the right panel.

docs.github.comAbout CITATION files - GitHub DocsYou can add a CITATION file to your repository to help users correctly cite your software.

docs.github.comReferencing and citing content - GitHub DocsYou can use third-party tools to cite and reference content on GitHub.

Preserve! Get a DOI! For free! Use Zenodo docs.github.com/en/repositor... or Dataverse github.com/marketplace/... or others I might have missed.

Add a proper citation. Authors prefer that you cite the article, because that's the only thing that counts for tenure....

But: if article is published in 2021, data in 2023, re-users are misleading their readers. The data they use doesn't exist at the cited location (the article, or its replication package), and cannot be easily linked. The solution is to cite both: article and Github.

For authors: All of the above can be greatly simplified if authors (those posting on Github) followed some simple best practices:

Version your key data. Please. Create a tag or release. It's easy.

The quote says "data are in the public domain" Wrong! Public domain is a legal term = nobody has the copyright on the data. Absence of true "public domain", the data are copyrighted! Strictly speaking, authors' permission to redistribute is needed. Or you might claim "fair use" explicitly.

BTW: We usually send authors who do this back to the original creators to ask for permission. Much cleaner. BBTW: This is Bluesky, and I am not a lawyer... None of this is legal advice!

On the Github README, the authors actually say The version released on Sept 29, 2023 is the latest. but the commits on the Github actually tell a different story: there are September and November updates to the deposit. Whether those are materially different is not clear.

aeadataeditor.github.ioaeadataeditor.github.io

The very verbose description is actually not the best way. Pins change, commits not so much. Here is the same verbose instructions expressed as a single URL: github.com/KPSS2017/Mea...

... To download, visit at github.com/KPSS2017, click the pinned repository named "Measuring-Technological-Innovation-Over-the-Long-Run-Extended-Data," and download "PatentSimilarityImportanceBreakthrough_forPost2022.csv.zip"." (BTW: Better layout? See aeadataeditor.github.io/posts/2026-0...

If using the Github data, and not the replication package to the original article, the Github data must be cited.

From a recent (draft) replication package: "Data on Kelly et al. (2021)'s patent indicator are downloaded from the authors’ GitHub repository. A copy of the data is provided as part of this archive. The data are in the public domain. ....

Recent case, re-run of the authors' code failed on a first-stage regression, blocked code from completing. Hard to initially diagnose, because "qui do regression_55.do" shows no output... 🤦Back and forth with authors, they provide evidence that it works on their system. We send MWE to Stata support👇

Lesson? I don't know about the econometrics - but don't use --quietly-- in #Stata ! (or "source()" without "echo=TRUE" in #R, or drop the ";" in #MATLAB). Good night, and good computing!

aeadataeditor.github.ioThe Same Code, a Different Answer: Stata Convergence Across PlatformsA recent replication came to a halt on a first-stage regression. The authors’ code ran fine on the authors’ computer. It failed on ours. Same code, same data, same software package — different answer....

8 Aug

Written up at aeadataeditor.github.io/posts/2026-0...

Of course, this was only 1 of many dozens of regressions in the paper - the rest all converged (how well, we did not get into - that remains for the authors to see, and future replicators!)

We only detect some of these issues - if we happen to have the same sweetspot hardware/software config as the authors, we will not find discrepant answers - , and they are hard for authors to diagnose, unless they have a similar museum of hard- and software versions as Stata does...

"If I write -shownrtol- option instead, I will see that (the 'minimum' of all tolerances) is 0, but exact zero means that something can't be computed, in this case the nrtol (which requires inverting the Hessian)."

(this is me again) Long story short, the Stata helpdesk did a fantastic job diagnosing the problem. But the gist is - know your regressions - don't use --quietly--

"2) There are huge confidence intervals,... coefficients are poorly identified." "3) The problem is the matrix of second derivatives, that is non invertible. In the cases where we get 'convergence', we get huge or missing standard errors, meaning that it can't really be inverted."

"On Mac, I don't get convergence by default, but I get it if I disable the -nrtolerance-, meaning that this is the tolerance that the model fails to meet."

"This is why we insist on the importance of examining the output and make sure that it makes sense. I looked at the output it shows several problems."

"1) there are missing standard errors. ... an indication that there is a problem with the Hessian (either it has missing values, or for some reason it can't be inverted), so you can't trust your results."

As it turns out: Authors ran with Stata 19 MP10 ! One of the few combinations that worked... I had sent a MWE (minimal working example - reducing the authors' problem from 100s of lines of code to 2 dozen). One of Stata's statisticians got back to me... I'm putting here much of what they wrote...

"Stata, like any other software package, has fixed criteria to convergence declaration. It's not foolproof. There will be always cases where convergence will be 'declared' and it shouldn't."

They respond very quickly, and then very thoroughly! Thanks @stata.bsky.social ! It remains... a mystery. While we ran the code on Windows Server (Stata 19now) and Linux (S19now), the authors ran it on Windows Enterprise (Stata19). Ours failed, theirs did not. Turns out, that's not enough.

Stata folks ran the following: "convergence on Windows box running Stata 18 MP32 and Stata 19 MP8, but `set processors` to any number less than 8 cause not converging....Mac mini Apple M2 pro/ S19MP16 NOT...Linux/S19MP16, S19SE, S18MP2, S17MP2, S16MP2, NOT converged for any number of processors"

14 July

From an anonymous replication package: the underlying ACS housing-unit microdata (~14 GB across 2006–2019) are **not** redistributed here because of their size. I understand that even back in 2006, 1.44MB floppy disks were going out of style... We draw the line at about 2TB, we have solutions!

26 June

I just wanted to remind folks: Excel! "In updating the replication package so that inputs for Figure 1 are produced by code... hard-coded input cell in the Excel workbook ... outdated reference" Luckily, caught, and change was minor. Please use code to produce tables, figures!

26 June

If you need to use Excel for figures (which can be easier, I admit), write to Excel from your programming language of choice. Then create the figure referencing the sheet or area that is filled in by your programming language, voilà: automatically updating figure!

larsvilhuber.github.ioHands-off running: Automatically saving figures — Self-checking replication packages

26 June

See larsvilhuber.github.io/self-checkin... for commands and examples.

aeadataeditor.github.ioCEA Summer School on Research ReproducibilityAbstract This intensive training explores the fundamental principles of reproducibility, a pillar of open science that ensures the transparency and credibility of scientific research. You will discov...

4 May

Also just a few spots available in VANCOUVER, with registration closing May 12, for #CEA2026 .

AEA Data Editor@aeadata.bsky.social

If French is not your thing, but 🇨🇦 is, come to Vancouver at the #CEA2026 for THAT summer school: aeadataeditor.github.io/talks/2026-0... @ubcvse.bsky.social @sfuecon.bsky.social (again with Marie Connolly, Data Editor for #CJE)