Johannes Breuer

Johannes Breuer

@johannesbreuer.com

Professor of Digital Social Science @unidue.bsky.social and head of the team Research Data & Methods @cais-research.bsky.social Interested in digital traces | computational social science | reproducibility | open science | #rstats www.johannesbreuer.com

2.9K followers2.4K following424 posts

Johannes Breuer reposted

doi.org/10.1162/REST... "Online experiments have surged in popularity, but how credible are their published results? We examine the universe of hypotheses tested on Amazon Mechanical Turk ... We find evidence of widespread p-hacking, publication bias, and under-powered studies."

Johannes Breuer reposted

Johannes Breuer reposted

Johannes Breuer reposted

Johannes Breuer reposted

New preprint! Universities increasingly expect academics to practise open science. But is it rewarded when hiring? We analysed 1,356 academic job ads from Norwegian institutions and found that <5% of ads mentioned open science as a required or desired qualification osf.io/preprints/ps...

Johannes Breuer reposted

Johannes Breuer reposted

Did you know? There are more than 50 panels in English open for paper proposals for the #dvpw2027 #Convention in Leipzig next year! Check them out 👉 www.dvpw.de/dvpw2027/en/... & submit your paper proposal by 31 October 👉 meine.dvpw.de/civicrm/kong... #PolSci #PoliticalScience #CfP

DVPW@dvpw.bsky.social

DVPW-Kongress 2027 #dvpw2027 "Welt im Umbruch. Politikwissenschaft auch?" 30. Wissenschaftlicher Kongress der DVPW, 7.-10.9.2027, Universität Leipzig Alle Infos 👉 www.dvpw.de/dvpw2027 #PoWi #Politikwissenschaft

I was in Vienna for a little over 24hrs and am very satisfied with my intake of "süße Mehlspeisen": I had Kaiserschmarrn, Apfelstrudel, and Topfenknödel. I am less satisfied with my coffee intake: I had no Melange, Einspänner or Kapuziner. Gives me yet another reason to come back...

Johannes Breuer reposted

Johannes Breuer reposted

Come work with us - we have cinnamon buns (and an amazing team, very nice colleagues and an inspiring interdisciplinary environment)

This post is unavailable.

Johannes Breuer reposted

🚨JOBS: Information influence campaigns are at the center of hybrid conflicts. As part of a new initiative on Hybrid War and Democracy that I lead at @d-ias.bsky.social, there are openings at post-doc and assistant professor (tenure track) levels. 🗓️: DL Nov 23 Share 🫶 🧵 www.sdu.dk/en/forskning...

sdu.dkVacant positions at the Danish Institute for Advanced Study (DIAS), University of Southern DenmarkDo you merge research excellence with immense curiosity and a sense of playfulness? We are currently hiring tenured DIAS Professors and tenure track DIAS Fellows across multiple disciplines - apply t...

Johannes Breuer reposted

Hi all! I'm looking for people to try out my new collaborative writing platform. Think: Overleaf or GDocs for Mkdown LaTeX HTML Typst Quarto Free, open source, and full-featured. Many editors, chat, comments, track changes, history, AI, mobile, templates, etc Would ❤️ any feedback! librepaper.org

Johannes Breuer reposted

📣 October News out! 🤩 Registration is open for #GESISspringseminar 2027 on Metascience: Methods and Tools for Robust Inference. 🔍 Interview with Cassie Ann Short & Nate Breznau on Metascience. ✨ Updates on #GESISworkshops. ⏪ Look back on #GESISsummerschool 2026. 👀 ➡️ t1p.de/Training-NL-....

Johannes Breuer reposted

Seit Sonntag kursiert online eine Grafik, mittels derer behauptet wird, ukrainische Drohnenangriffe auf Rechenzentren des russischen Konzerns Yandex hätten zu einem Rückgang russischer Bot-Aktivitäten bei Desinformationskampagnen rund um die AfD geführt. Dieser Zusammenhang ist bislang unbelegt.

br.de#Faktenfuchs: Irreführende Grafik zu angeblicher Bot-AktivitätIm Netz kursiert eine Grafik, die angeblich zeigen soll, wie infolge von Angriffen auf russische Rechenzentren die Online-Aktivität rund um die AfD abgenommen habe. Doch die Grafik weist große Unstimm...

Johannes Breuer reposted

renebekkers.wordpress.comFast, but incomplete and inaccurate: twelve AI-powered tools for literature searchThere seems to be s a lot of enthusiasm for AI powered tools for research these days, if I look at the feeds that the tech industry controls. No doubt that these tools greatly reduce the effort for a literature review that sounds plausible. It’s great to get a nice set of studies as a starting point for a new project in a new research area. But in these cases we usually don’t know what we don’t see. So here’s a cautionary tale from a rare case where we do know the errors of omission in AI powered tools. The challenge I gave the AI-powered tools is difficult: I asked them to identify replications of previous experiments in the multidisciplinary field of research on charitable giving. Such replications are needles in a field of haystacks. But we found them by hand, with a multidisciplinary team of six scholars in a study on replications of research on charitable giving. We conducted a comprehensive manual search on Google Scholar and identified 48 replications of experiments. How many replications from the set of 48 can AI-powered tools identify? Not many, it turns out. The best tool, Google Scholar Labs, found one third of them. Most tools – even popular ones such as Perplexity, find fewer than 10. LeapSpace, the tool that Elsevier is trying to sell to university libraries in the Netherlands, performed abysmally. Collectively the twelve tools I assessed identified exactly half of all replications that we had found by hand. But it gets worse. Many studies that the AI tools identified as replications were not replications at all. Prompting more elaborately with explanations and examples did not make the results better. The news is not all bad. The tools also identified a number of new studies that we had not found because we had limited our search to keywords that returned a feasible number of results. The newly found replications used other terms than we had looked for. Another upside: at least the searches came back quickly. The slowest tool, Claude Science, took 35 minutes. The quickest tool returned results within a few seconds. The result: it could not find any of the replications. The results underscore the necessity of human review of results from AI-powered tools. At face value, the reports produced by the tools seem entirely plausible. However, human review of the results showed a high number of false positives, up to 70% of all results for LeapSpace. Conclusions based on the searches of the AI-powered tools are incomplete and incorrect. Without an accurate and efficient method to identify false positives the speed with which the tools concluded their work gives users the false impression that they can outsource identifying relevant previous work to AI-powered tools. Read the full paper including the prompts and supplementary materials with all details and a link to the data and code here. Like Loading... ### _Related_

"Teil der Zersetzungsstrategie der AfD. Es ist wie in dem Kartenspiel „Werwolf“, bei dem eine Dorfgemeinschaft den unbekannten Werwolf sucht, der nachts heimlich und anonym mordet, aber tagsüber versucht als vermeintlich harmloser Dorfbewohner mit Lügen und Falschbeschuldigungen Zwietracht zu säen."

taz@taz.de

Zum ersten Mal seit 1945 wurde in Magdeburg ein Rechtsextremist zum Präsidenten eines deutschen Parlaments gewählt – und CDU-Abgeordnete lügen über ihr Wahlverhalten. 👉️ taz.de/!6216651/

Johannes Breuer reposted

Postdoctoral Researcher in Computational Social Science (NLP/ML) #ECPRMemberJobs 🎓 A PhD in computational social science, computer science, data science, or political science with strong computational methods 🏫 Koç University ⌛ Sunday 1 November 🔗 buff.ly/pAuDNia #PolSciJobs #PoliSky #AcademicSky

buff.lyPostdoctoral Researcher in Computational Social Science (NLP & Machine Learning)Shape the future — Koç University

Johannes Breuer reposted

Johannes Breuer reposted

We're excited that Leah von der Heyde will join us for the next session of the TaDa Fall Speaker Series: “How (Not) to Use LLMs in Survey Research” When? Wednesday, October 14, 17:00 (Berlin time) Sign up for our newsletter at tada.cool for the zoom link!

tada.coolTaDa — Text-as-Data Speaker SeriesFree online talks on text-as-data, NLP and machine learning in the social sciences. Biweekly Wednesdays, 17:00 Berlin time, on Zoom.

Johannes Breuer reposted

🧵 Just accepted @thejop.bsky.social, with @tswalve.bsky.social "What's in a name? Name recognition, place-based identities, and candidate success" Voters prefer local candidates. But how do they know who's local? Often simply from the name on the ballot👇 www.journals.uchicago.edu/doi/10.1086/...

journals.uchicago.eduWhat's in a name? Name recognition, place-based identities, and candidate success | The Journal of Politics: Vol 0, No ja

My 9 games #My9Games my9games.net/en/share/8f5...

Dr. Rachel Kowert@drkowert.bsky.social

Nine games that changed who we are? There are so many more than 9... but as you can tell from the image, Final Fantasy has changed who I am foundationally as a human being (It's ridiculous that Zelda isn't on my list or Witcher 3 isn't here... but I started picking CHRONOLOGICALLY... Geralt 4 Life)

Johannes Breuer reposted

What do we expect of robustness reproductions in the age of AI? Help us find out at the Münster Reprohack workshop from December 2-4. We will conduct AI-led and human-led robustness reproductions, that are subsequently submitted to @r2journal.bsky.social Apply at indico.uni-muenster.de/e/reprohack

indico.uni-muenster.deReproHackReproHack: Join us for a mentored three-day reproduction hackathon! Background When building on past research, we rely on it being correct and robust - and yet, these characteristics are rarely assess...

Johannes Breuer reposted

We're hiring research engineers, research associates & several postdocs at the Wisdom & Culture Lab @UWaterloo to build experiments, tools & models on how AI shapes human reasoning, decisions & culture. Psych, cogsci, ML/NLP, HCI & eng backgrounds welcome. uwaterloo.ca/wisdom-and-c...

Johannes Breuer reposted

Johannes Breuer reposted

Watch a recording of our latest methods #webinar on implementing generative artificial intelligence into #survey processes. Patrick Sturgis (@lsemethodology.bsky.social 🇬🇧) presented the results from two studies on the use of #AI for questionnaire pretesting and occupational coding. buff.ly/T1pMTsf

buff.lyGenerative AI and the future of survey measurementPatrick Sturgis (London School of Economics) presented a survey methods webinar about using generative AI in surveys on 15 September 2026. Large language models have the potential to transform many…