Sharad Goel

Sharad Goel

@5harad.com

Professor of Public Policy at Harvard, co-director of @comppolicylab.bsky.social, applying a computational approach to public policy, including to issues in education, healthcare, and criminal justice. https://5harad.com

1.3K followers204 following15 posts

Sharad Goel reposted

Sharad Goel reposted

In a large field experiment with real students, o3-mini (very obsolete) in an agentic loop made very good exam questions: "we found that this approach achieves psychometric properties on par with those of questions appearing on high-stakes standardized tests." arxiv.org/pdf/2508.08314

Sharad Goel reposted

Standardized test scores improve predictions of academic performance but matter relatively less for admissions decisions, since few applicants are near the decision threshold, from Johann D. Gaebler, Calvin Isley, Christopher Avery, and Sharad Goel www.nber.org/papers/w35407

📣 Job alert! 📣 Harvard Kennedy School is running 2 open-rank faculty searches in technology & public policy. One position is specifically for scholars using data science to address public problems. Please help us spread the word! academicpositions.harvard.edu/postings/15561

academicpositions.harvard.eduTechnology and Public Policy Search (open rank)Harvard Kennedy School invites applications for two open rank appointments in technology and public policy. We seek applications from scholars whose research and/or practice focuses on the intersectio...

Sharad Goel reposted

NEW in Management Science! My coauthors and I came up with a new consequentialist approach to designing equitable algorithms. Instead of imposing fairness criteria on an algorithm (like equal false negative rates), we aim for good outcomes. More in the 🧵 below! (1/)

View full thread

As a result, the overall clinical utility of using race in these models is often surprisingly small. It still might make sense to include race, but the benefits of doing so have probably been overstated.

There are, of course, lots of important caveats to this work. For example, the results change if we move to a resource-constrained setting. But we hope our work illustrates the importance of looking at utility, not simply model miscalibration, in these debates.

View full thread

But despite this miscalibration, *decisions* (e.g., to intervene in some way) based on race-aware vs. race-unaware risk estimates are largely the same, as very few individuals are close to the decision boundary.

And because people for whom decisions flip are necessarily close to the decision boundary, their *utility* for intervening vs. not is comparable. (In the shared decision-making settings we consider, the utility is 0 at the boundary, i.e., the boundary is set to be the point of indifference.)

We find that race-unaware risk estimates are indeed often *miscalibrated*, systematically over or underestimating risk for different groups. Such miscalibration is often cited as evidence that including race improves the quality of predictions for all groups.

Sharad Goel reposted

Calling instructors at 2- and 4-year colleges! Give your students free access to a virtual tutor optimized for learning by taking part in our study on using AI to improve education. Learn more and sign up by Dec. 16: zurl.co/WitA

zurl.coPingPong 🏓
Sharad Goel@5harad.com

Instructors at 2- and 4-year colleges! Give your students free access to a virtual tutor optimized for learning by taking part in our study on using AI to improve education. Instructors receive a $1,000 honorarium. Sign up by Dec. 16! pingpong.hks.harvard.edu/eduaccess

Our virtual tutor, called PingPong, is based on ChatGPT. It's designed to help students learn, not simply give them answers to homework problems. We'll help customize PingPong to your course content. Instructors can view de-identified student interactions. PingPong is multilingual.

Naviance is widely used by high school students to help them decide where to apply to college. In our new PNAS paper, Sabina Tomkins, Josh Grossman, Lindsay Page and I show it can inadvertently dissuade qualified students from applying to selective schools. www.pnas.org/doi/10.1073/...

Our paper "The Measure and Mismeasure of Fairness" — long in draft form — is now out in JMLR! We show that common error rate measures are often misleading indicators of algorithmic bias, and argue it's better to evaluate algorithms by looking at their effects. 5harad.com/papers/fair-...

We find that Asian American applicants — especially South Asians — are much less likely to be admitted to selective colleges than white students with similar test scores, GPAs, and extracurricular activities. Much of the gap is due to geography and preferential treatment of legacy applicants.

Josh Grossman@jdgrossman.com

There is considerable debate over whether Asian American students are admitted to selective universities at lower rates than similar white peers. In new work with Sabina Tomkins, Lindsay Page, and @5harad.com, we explore this issue using nearly 700k college applications. 🧵 nber.org/papers/w31527

You are up to date.