Ben Tappin

👇New experiments in which we aimed to map the levers and scope of political persuasion with conversational AI models. It was a tremendous privilege to lead on this work alongside the brilliant @kobihackenburg.bsky.social. The paper is packed with results and we'd love your comments!

New paper! Despite qualitative researchers claiming that generalizability is not a goal, their published work does indeed contain generalizable claims. We document this and provide recommendations for how all researchers can better calibrate their generalizability claims. doi.org/10.1177/2515...

Extremely impressive field experiment by @nmalhotra.bsky.social et al. had few effects on election attitudes among conservatives in TX despite being "exposed to an average of 20 online videos, 18.5 cable TV videos, 23.9 display ads, 3 text messages, and 3 postcards" www.science.org/doi/full/10....

Back from my summer break with a piece for @transformernews.ai on why the impact of persuasive AI's will be smaller in the real world than some think because (1) it is hard to reach people and get their attention… buff.ly/TvWgFSH

After six years as Head of the Department of Psychological and Behavioural Science, Professor Liam Delaney is handing over the role to Professor Alex Gillespie. We're grateful to Liam for his leadership and delighted to welcome Alex as our new Head of Department. Learn more 👇️

Psychological Science special issue call 📣 Ever had your research denigrated bc it's "only descriptive", "what's the mechanism though?" and "but what are the theoretical implications?" Now has come your time to shine. Send your best descriptive work! www.psychologicalscience.org/publications...

As promised, we now share some details why it took 32 months to get this Matters Arising published, and, since many asked, share some comments on the reply to our critique. All details can be found in this small FAQ vincentholst.github.io/the_curious_.... Below is a summary 🧵

There are still a few days to apply to work with me and @malte.the100.ci as a PhD or postdoc on the development and evaluation of RegCheck. Help us research whether RegCheck works in practice in helping to reduce preregistration-paper discrepancies. Apply below!

New paper out now 🥳 When psychologists discuss generalisability, they often refer to vague notions of representativeness. We provide an accessible intro to the total survey error framework as a tool to reason about this more rigorously. w @taymalsalti.bsky.social @ruben.the100.ci >

Important @benmtappin.bsky.social on the need to carefully consider the relevant counterfactual when evaluating AI chatbots (also applies to social media!) benmtappin.substack.com/p/are-ai-cha...

Really helpful framework for thinking about the utility of survey experiments for practitioners by @benmtappin.bsky.social 👇 www.benmtappin.com/publication/...

How do you align AI in a world of plural, conflicting, and evolving human values? A starting point is human society itself. @sydneylevine.bsky.social and I are hiring a postdoc at NYU to combine insights from cultural evolution, computational moral cognition, and AI safety. Please share widely!1/

I wonder how these rates of sycophancy (and their effects) compare against realistic counterfactuals like talking with one’s close friends or spouse.

The synth backing track takes the vibe to an unexpected place 🥲

Closing out the teaching semester this week with one of my favourite topics. Sadly its pedagogy was forever and unforgivably mar'd by the truly worst naming convention of all time...

Yes. Then if you push on this the claim becomes “okay it may not cause but it’s still useful because it predicts”. But then it’s like if predictive accuracy was your goal the design and analysis should be different e.g., CV + more predictors. I’ve been fully Westfall & Yarkoni-pilled on this point.

Sydney named its socio-economic divide the "latte line" and has been arguing about who drew it for 20 years. London has the same divide and calls it "character." I built a machine learning model to do the impolite thing: draw it and blame someone. open.substack.com/pub/laurenle...

someone at the pentagon frantically typing “Claude, open the strait of Hormuz for me, quickest possible strategy, make no mistakes.”

The character of Per’s posts increasingly remind me of when an LLM talks to itself (not derogatory)

The idea of the need for a control group may have been radical at the time, but trust me it's now generally accepted. This is not some fringe believe by methodological hardliners; it's established practice. www.clinicaltrialsabundance.blog/p/clinical-t...

New newspaper headline for your Intro to Causal Inference lecture just dropped

Planning my next conversation starter

I too was very glad to see this! But I feel like the whole episode bodes badly for the future. It’s not sustainable to rely on the CEO of a private company to act against their financial self-interest in order to curtail high-risk AI deployment (here mass surveillance and fully autonomous weapons).

🔔 “How real is the LLM threat to online research in academia?” will be live today. Experts from Microsoft Research, MIT / Stanford, Max Planck Institute, and Prolific discuss the threat of agentic AI to online research, and how to protect against it. Link to join live below. #AcademicSky #Research

Reposting for visibility. Many researchers still appear oblivious to this fact, which is terrifying! It should be included in every experiment design 101.

When I pitch academics on my paper on nulls one common and understandable reaction is "but they're probably noisy and thus uninformative nulls." This is true, but it misses the key realization that WE PUBLISH THE RESULT WHEN THE NOISY TEST IS P<0.05.

I have a new paper. We look at ~all stats articles in political science post-2010 & show that 94% have abstracts that claim to reject a null. Only 2% present only null results. This is hard to explain unless the research process has a filter that only lets rejections through.

@benmtappin.bsky.social I just was pointed to this which is much more thorough and arrives at the same conclusion: www.exponentialview.co/p/how-95-esc... The "95% fail" number is essentially meaningless

Felix and friends looking closely at the details so you don’t have to 👌👇

A short note on questionable AI studies or why friends don’t let friends make %-claims based on small-n qualitative research interview reports New week, new AI newsletter from Marina and myself here at RISJ: buff.ly/ckaUSn9

It’s that time of year again

🚨New WP "@Grok is this true?" We analyze 1.6M factcheck requests on X (grok & Perplexity) 📌Usage is polarized, Grok users more likely to be Reps 📌BUT Rep posts rated as false more often—even by Grok 📌Bot agreement with factchecks is OK but not great; APIs match fact-checkers osf.io/preprints/ps...

My Centre is a unique place to do a PhD in Philosophy, because you can be in constant contact with experts in veterinary medicine, psychology, zoology and policy and be part of a team united by a shared interest in animal minds. We now have our 1st ever PhD scholarship: www.lse.ac.uk/sentience/phd

📢 JOB ALERT! Postdoc opportunity in Political Behaviour & Political Economy (UKRI‑funded) - Please do share with anyone who might be a great fit. If you’re interested or would like to know more, please feel free to get in touch!! jobs.reading.ac.uk/Job/JobDetai...

🎺 Call for proposals 🎺 1️⃣ replicate an existing experiment 2️⃣ run a novel experiment on repdata.com 3️⃣ coauthor with Mary McGrath and me to meta-analyze the replications and existing studies 4️⃣ publish your study details: alexandercoppock.com/replication_... applications open Feb 1 please repost!

I land somewhere between this and the OP. Evaluating the quality of the methods often requires fully understanding the research question and estimand. And that usually requires reading the intro. (Disclaimer: but even then it’s no guarantee😭 cf. www.the100.ci/2024/08/27/l... @dingdingpeng.the100.ci)

My students are in for a treat next week

A very interesting data-rich analysis of persuasion on digital media by @benmtappin.bsky.social. Recommend! open.substack.com/pub/benmtapp...

🚨 New in Nature+Science!🚨 AI chatbots can shift voter attitudes on candidates & policies, often by 10+pp 🔹Exps in US Canada Poland & UK 🔹More “facts”→more persuasion (not psych tricks) 🔹Increasing persuasiveness reduces "fact" accuracy 🔹Right-leaning bots=more inaccurate

🚨 New working paper 🚨 We often see populist parties like Reform UK blame higher energy bills on climate change policies. What are the political consequences of this strategy? Very early draft; comments and criticisms are welcomed! full draft: z-dickson.github.io/assets/dicks...

"While testing one dimension at a time can yield simple results, those effects may not generalise to richer, real-world contexts." Read our new POAL Methods Briefs on Conjoint Experiments from Thomas Robinson! Link: www.poal.co.uk/research/met...

Insightful long-read: "With AGI [artificial general intelligence], powerful actors will lose their incentive to invest in regular people–just as resource-rich states today neglect their citizens because their wealth comes from natural resources rather than taxing human labor." intelligence-curse.ai

🗞️ 🤖 Weekend reading anyone? For the launch of @transformernews.ai as a standalone publication, they invited me to contribute a piece on what persuasive AI might mean for democracy and elections. Here’s the result… buff.ly/OJsNmpK

WE ARE HIRING! 2 Lecturers in Quantitative Social Science. Want a friendly interdisciplinary department in one of the world's most vibrant cities? This just might be for you. Apply by: 10 Oct www.ucl.ac.uk/work-at-ucl/...

Wrote about how the UK's online age verification requirements are already proving to be a disaster (as UK regulators were clearly warned it would be) and how unhelpful the UK's response to this mess has been, including their tech minister saying anyone who complains supports predators.