Post

Had a small part of this that I am pretty proud of: Full push-button reproducible manuscripts from data to manuscript. Nearly every stat, figure, number, table, etc is pulled directly from analysis code and data and "knit" with manuscript text written in Google Docs. Blog post coming soon ...

Code and data (both for the analyses and for the knit itself) are of course available and linked in the papers. For the paper on reproducibility, see here osf.io/ed8pj/overview For the paper on replicability, see here osf.io/g5sny/overview Code/data are CC0 1.0 Universal, enjoy!

Reproducible manuscripts are useful for a few reasons: One of the best reasons is just practicality. The thing most of us do where we have to manually update every stat/figure etc in a manuscript every time something changes in the code or data is super error-prone and time consuming.

Then there is rigor and accountability. Anyone can trace every stat and figure to *exactly* how it was estimated. If there's an error or a way you think things should have been done instead, it's super findable. As a bonus, that incentivizes code readability, since people might use it.

Reproducible manuscripts have been around since code existed, but they are still pretty rare and mostly written in scripting langauges (text and everything) like RMarkdown. For SCORE, the text is written collaboratively via Google Docs, enabling modern collaborative writing with a large team.

I hope y'all tear into it and find any lurking issues, questions, ideas for your own work, etc. Gotta say it's a bit terrifying to have the entire pipeline so fully exposed and accessible, particularly one as complicated and high profile as SCORE. But more importanty, I hope it's useful.

update: internal muffled scream

Profoundly weird to see these charts in the science mainstream, but if you want to remake them it's all in the code side note: the color palettes for the above linked reproducibility and replicability papers were based on samples from Weezer album cover art. you're welcome.