Donald Bowen

WP thread: “Measuring and Mitigating Racial Bias in LLM Mortgage Underwriting” with Don Bowen, McKay Price, and Ke Yang Our audit study asks AI to assess simple mortgage applications (real HMDA data w/ randomized race and credit scores) 🦶🦶🏼🦶🏿 Download lukeste.in/llmmortgage 1/

Donald Bowen reposted

Josh Kaplan

@josh-kaplan.bsky.social

4 Jan 2025

THREAD: In 2023, I received an envelope with no return address. Inside was a flash drive containing tens of 1000s of secret files. It came from a vigilante with a tumultuous past, who'd conducted a years-long undercover operation. He didn’t tell the FBI or his family. He only told me.

🧵 I'm happy to share an updated draft! "Revisiting Board Independence Mandates: Evidence from Director Reclassifications" provides causal evidence on the effects of mandated board independence. Joint with Jerome Taillard (@babsoncollege.bsky.social) #ASSA2025 #EconSky #FinSky 1/N

This paper is an example of how nice it is to work with great coauthors AND how great peer review can be. (Weird? But true!) We got wonderful feedback from referees at the JFQA previously and our current editorial team, and these have materially improved the paper. We are very grateful. 10/

papers.ssrn.comRevisiting Board Independence Mandates: Evidence from Director Reclassifications<div> We provide causal evidence on the effects of mandated board independence. We compare firms that replace existing non-independent directors to firms that

Much more in the paper (linked) I'm not at #ASSA2025 (new kids) but able and willing to connect otherwise! 11/11

Mechanism: Reclassified directors had ~25% more board experience & were 60pp more likely to be former employees. When firms couldn't reclassify, new directors focused more on monitoring (20% more likely to join audit committees) vs advising. 8/N

Policy implication: One-size-fits-all governance mandates can harm firms by forcing out directors with valuable firm-specific knowledge. More flexible approaches might better balance monitoring & advising roles. Firms know this & recognize value of their dirs and so reclass when possible. 9/N

Key findings: Non-reclassifying firms saw ROA decline by 2.7pp vs reclassifiers. Main channel? Labor efficiency. These firms had higher SG&A costs, lower sales/profits per employee, and their 10Ks discuss process innovation less -> 📉 of op. expertise as knowledgeable directors depart 🛫 6/N

Design insight: Previous work compared compliant vs non-compliant firms. This paper's triple-diff compares reclassifying vs non-reclassifying firms within the non-compliant group. Cleaner identification. 7/N

Identification: Some firms could reclassify directors as "independent" (eg if they retired > 3 years ago) while others had to hire new outside directors. Reclassification eligibility was largely predetermined before mandates > quasi-random variation in compliance strategy that we can exploit. 4/N

Reclassifying firms (R=1) became compliant with exchange independence quickly via this way (panel 2), even though ISS classifications of their directors remain least independent (panel 3). Upshot: R firms are able to retain employee directors at much higher rate (panel 4) 5/N

Punchline: After the 2002 NYSE/NASDAQ board independence mandates, firms that replaced existing non-independent directors underperformed firms that retained these directors by reclassifying them as independent. Suggests valuable firm-specific knowledge was lost. 2/N

Takeaways 1. Inside dir advice is performance relevant via oper. efficiency 2. Insiders <=/=> agency problems 3. Gov standards disregard pre-existing governance arrangements (which tradeoff monitor/advice) 4. Speaks to policy debates (eg the push for maximal independence) 3/N

18 Dec 2024

@emollick.bsky.social I thought this little trick mighty be of interest

Donald Bowen@bowen.finance

Somehow, just asking LLM to be unbiased • Eliminates approval recommendation gap (on average and across different credit scores) • Reduces average racial interest rate gap by about 60% (from 35bp to 14), with even larger effects for lower-credit-score Black applicants 8/

17 Dec 2024

Can't wait for the @khoavuumn.bsky.social meme on this paper

Mihail Velikov@mvelikov.bsky.social

An academic paper has excellent empirical evidence & hypotheses that perfectly match the patterns in the data. One catch: AI wrote the hypotheses after seeing the results. Should this matter? New paper w/ Robert Novy-Marx on AI-Powered (Finance) Scholarship🧵 papers.ssrn.com/sol3/papers....

17 Dec 2024

Really cool work that raises questions about how we think about science and progress. The push towards pre registration has benefits but foregoing HARKing has many costs too. The optimal balance is not obvious!

Mihail Velikov@mvelikov.bsky.social

lukeste.inMeasuring and Mitigating Racial Disparities in Large Language Model Mortgage UnderwritingWe conduct the first study exploring the application of large language models (LLMs) to mortgage underwriting, using an audit study design that combines real lo

My coauthors (@lukestein.com, McKay Price, and Ke Yang) and I were thrilled to find out our work “Measuring and Mitigating Racial Disparities in LLM Mortgage Underwriting” was recognized as the Best Paper at the (fantastic!) New Zealand Finance Meeting acfr.aut.ac.nz/conferences-...

There’s more in the draft lukeste.in/llmmortgage Obviously no one should (/would?😢) make critical financial decisions using prompts so simple. But genAI is getting deployed and importance of audits and intentional design are even more important in more complex systems! Fin.

Thanks and apologies to @lukestein.com for the thread copy. Can't improve on this!

(Training data incl. long history of racial disparities in mortgages, plus prompts may trigger bias learned elsewhere) We give access to explicit race information, which should make it easy to 𝘢𝘷𝘰𝘪𝘥 discrimination if LLMs can ignore it, as they know they should… 7/

LLM’s mortgage racial bias biggest for applications with lower credit scores [or high DTI, high LTV]: • The Black–white approval rate gap is ~56% greater for low-score applicants than at average (13.3pp vs. 8.5) • Interest rate gap ~32% greater (47bp vs. 35) 5/

We find anti-Black bias in mortgage underwriting recommendations from a number of LLMs Black borrowers with low credit scores suffer the most 6/

I.e., ChatGPT (we also assess other models) • Claims it’s unbiased… • …but it 𝗶𝘀 biased against Black applicants. Particularly at low credit scores. (We also have this for other risk measures.) • We can partly close the racial gap just by asking for unbiased responses. 3/

LLMs recommend denying more loans and charging higher interest rates to Black applicants They would, on average, need credit scores ~120 points higher than white applicants to receive the same approval rate; ~30 higher to get same interest rate 4/

LLMs recommend denying more loans and charging higher interest rates to Black applicants Especially at low credit scores/riskier loans Simple prompt engineering can help mitigate gaps