Katie Keith

Katie Keith

@katakeith.bsky.social

NLP and computational social science (CSS) researcher. Assistant Professor in Computer Science at Williams College. AI2 and UMass Amherst alum. she/her. https://kakeith.github.io/

4.1K followers290 following35 posts

Katie Keith reposted

Andy Halterman
@ahalterman.bsky.social

Very excited that my paper with @katakeith.bsky.social is now out in @polanalysis.bsky.social. We investigate whether LLMs actually follow the instructions/definitions provided in codebooks, propose some diagnostics, and release a new evaluation dataset. www.cambridge.org/core/journal...

cambridge.orgCodebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts | Political Analysis | Cambridge CoreCodebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts
Katie Keith
@katakeith.bsky.social

Topic @adeldaoud.bsky.social and I were discussing today at lunch at #ic2s2 and want to ask here: What are the “known facts” in the social sciences? Which relationships between at least two social variables have been empirically found to have large effects and replicated by multiple groups?

Katie Keith
@katakeith.bsky.social

Highlighting this thread. Based on what I'm seeing at #ic2s2 this week, this line of work is hot (if a bit crowded), but I predict will only be more widely adopted by social scientists in the future.

Maria Antoniak@mariaa.bsky.social

What are your favorite recent papers on using LMs for annotation (especially in a loop with human annotators), synthetic data for task-specific prediction, active learning, and similar? Looking for practical methods for settings where human annotations are costly. A few examples in thread ↴

Katie Keith
@katakeith.bsky.social

We used active learning to create a human-annotated dataset of 1050 instances from FOMC transcripts—labeled for FOMC members’ opinions and directional stance towards monetary policy. Preprint and dataset should be released publicly by the end of the summer but email me for an advanced copy.

Katie Keith
@katakeith.bsky.social

For settings with an unobserved (but known) confounding variable, we propose a new causal inference method that uses two instances of pre-treatment text data, infers two proxies using two zero-shot models on the separate instances, and applies these proxies in the proximal g-formula.

Katie Keith reposted

Dallas Card
@dallascard.bsky.social

I'm excited to share that we've released v1.0 of our podcast corpus, SPoRC, led by my PhD student Ben Litterer! This first dataset is a slice of time, comprising over one million episodes from May and June 2020, including transcripts, diarization, and extracted audio features.

Ben Litterer @blitt.bsky.social

Podcasts are a popular medium, but data for computational research is limited! We introduce the Structured Podcast Research Corpus (SPoRC - huggingface.co/datasets/bli...), a large, multimodal dataset of English podcasts 🧵 arxiv.org/abs/2411.07892

arxiv.orgMapping the Podcast Ecosystem with the Structured Podcast Research CorpusPodcasts provide highly diverse content to a massive listener base through a unique on-demand modality. However, limited data has prevented large-scale computational analysis of the podcast ecosystem....

Katie Keith reposted

Mubashar “Mubs” Iqbal
@mubashariqbal.com

Starter packs are genius, but I was surprised there wasn't a list of them for people to find. So I built it: blueskydirectory.com/starter-pack... The website monitors the packs being shared and adds the ones it finds to the database. Missed your stater pack? Message me and I'll get it added.

blueskydirectory.comAll - Bluesky DirectoryA curated collection of all things relating to the Blue Sky social media platform.

Katie Keith reposted

Marc Keuschnigg
@marckeuschnigg.bsky.social

We are hosting the 11th International Conference on Computational Social Science in Sweden 🚀The IC2S2'25 website is LIVE, and submissions are OPEN! 📍Norrköping | July 21-24, 2025 Call for Abstracts (until Feb 24) Call for Tutorials (until Jan 17) 🔗Explore details & submit: ic2s2-2025.org

View full thread

You are up to date.