Peter Henderson

Peter Henderson

@peterhenderson.bsky.social

Assistant Professor, leading the Polaris Lab @ Princeton (https://www.polarislab.org/); Researching: RL, Strategic Decision-Making+Exploration; Law

3.8K followers559 following261 posts

I suspect much of the public would react very differently to AI (and toward AI co employees) if it was built like the NASA moon-landing, focusing on hard public good projects and by civil servants. 🤷‍♂️

🚀Last week we announced the launch of a new AI tool in partnership with the New Jersey Office of the Public Defender! AI can empower civil servants and improve public services, and this project provides a blueprint for doing so responsibly. 👇Lots of takeaways, below!

Dominik Stammbach@dominsta.bsky.social

How can AI support public defense? We have worked on this question for the last two years, and have (1) built an AI retrieval tool for the NJ Office of the Public Defender and (2) conducted qualitative interviews with defenders. Our main insights 👇(1/12)

View full thread

This work would not be possible without @dominsta_nlp , in partnership with NJOPD, the NJ Innovation Authority, the NJ Office of Information Technology, AWS engineers, and public defenders who shared their expertise.

🚀Last week we announced the launch of a new AI tool in partnership with the New Jersey Office of the Public Defender! AI can empower civil servants and improve public services, and this project provides a blueprint for doing so responsibly. 👇Lots of takeaways, below!

Dominik Stammbach@dominsta.bsky.social

How can AI support public defense? We have worked on this question for the last two years, and have (1) built an AI retrieval tool for the NJ Office of the Public Defender and (2) conducted qualitative interviews with defenders. Our main insights 👇(1/12)

View full thread

These new evals also showed that most open-weight embedding models fall short of being useful in this context... so we ended up fine-tuning models using synthetic data to shore them up for this real-world use case!

Overall, I'm excited to continue to find ways to do AI research while helping empower civil servants and improve public services. As a first step, we're excited to see what NJOPD, other offices, and the public do with the resources we've created!

🚀Last week we announced the launch of a new AI tool in partnership with the New Jersey Office of the Public Defender! AI can empower civil servants and improve public services, and this project provides a blueprint for doing so responsibly. 👇Lots of takeaways, below!

Dominik Stammbach@dominsta.bsky.social

How can AI support public defense? We have worked on this question for the last two years, and have (1) built an AI retrieval tool for the NJ Office of the Public Defender and (2) conducted qualitative interviews with defenders. Our main insights 👇(1/12)

View full thread

Before doing this we conducted structured interviews to identify concerns. One identified risk related to hallucinations. As a result, we focused on an extractive-first system, with optional generative summaries for only the facts of the case to situate the retrieved content.

To evaluate our system we collected a new benchmark of representative queries, with the help of real appellate-level OPD attorneys—we're immensely grateful for their time! These queries are often different from existing academic benchmarks and help make evals more real!

🚀Last week we announced the launch of a new AI tool in partnership with the New Jersey Office of the Public Defender! AI can empower civil servants and improve public services, and this project provides a blueprint for doing so responsibly. 👇Lots of takeaways, below!

Dominik Stammbach@dominsta.bsky.social

How can AI support public defense? We have worked on this question for the last two years, and have (1) built an AI retrieval tool for the NJ Office of the Public Defender and (2) conducted qualitative interviews with defenders. Our main insights 👇(1/12)

Public defenders manage large caseloads and tight deadlines. They also rely on a wealth of shared knowledge: past briefs, memos, directives, and lessons from colleagues about what has—and hasn’t—worked in court. Here's an example of an AG directive memo.

Until now, finding the right material across that collection could take hours. The AI-enabled Resource Library lets NJOPD staff enter a legal question and quickly find relevant passages from a secure, closed collection.

Btw, we actually covered almost exactly this scenario in our piece a few years back. But: negligence and products liability are (maybe) shifting—at least for co's that take few safety precautions—so there may be some updating needed. www.journaloffreespeechlaw.org/hendersonhas...

Peter Henderson@peterhenderson.bsky.social

This is wild, but really great investigative journalism. I guess all the jailbreaking papers trying to prevent "How do I build a bomb?" queries have been vindicated... So much for it being an unrealistic scenario... www.nytimes.com/2026/07/10/u... casp.ac/reports/ai-e...

Peter Henderson reposted

How can AI support public defense? We have worked on this question for the last two years, and have (1) built an AI retrieval tool for the NJ Office of the Public Defender and (2) conducted qualitative interviews with defenders. Our main insights 👇(1/12)

You can also check out our website and explore the work in an interactive way! Pick which question you think is better from justices or models, see the coverage of different legal issues from various models, & more! princeton-polaris-lab.github.io/oral-args-we...

princeton-polaris-lab.github.ioAI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments
Peter Henderson@peterhenderson.bsky.social

The future of oral argument or trial prep will obviously involve AI. But how can we measure quality of simulations? We build out an eval suite and run some studies! Sycophancy & issue coverage are clear areas of improvement, but models are surprisingly useful! 👇👇

The future of oral argument or trial prep will obviously involve AI. But how can we measure quality of simulations? We build out an eval suite and run some studies! Sycophancy & issue coverage are clear areas of improvement, but models are surprisingly useful! 👇👇

skyien-z.bsky.social@skyien-z.bsky.social

@nealkatyal.bsky.social says he practiced for Supreme Court oral arguments using #HarveyAI. But does AI as an oral argument simulator live up to the hype? In a new paper, we present the first comprehensive framework for evaluating AI as a practice partner for oral argument preparation.

It's worth understanding the many tools the federal government has to basically take out a frontier lab unilaterally—and the jawboning that this may cause. Hopefully a useful starting point for mapping out additional authorities beyond export controls.

Peter Henderson reposted

The rule against viewpoint discrimination is one of the most imptl in First A law. But as I show in a new paper, forthcoming in the U Penn Law Review, the test of viewpoint discrimination has changed a LOT in the past few decades, in good ways and bad. 🧵 papers.ssrn.com/sol3/papers....

papers.ssrn.comThe New Law of Viewpoint Discrimination<p><span>The prohibition against viewpoint discrimination is one of the oldest and most important principles of First Amendment law. But what it means to viewpo

Forcing people to leave the country to transfer to a green card, will hurt families, communities, and United States innovation. If you aren't aware, adjustment of status can take many months—if not years. I hope this policy is reconsidered.

Current discussions on AI's labor impacts seem to glaze over: How long will the transition take, in our lifetime? Will wages be the same? Will power be more/less concentrated? Will peoples' new jobs be as satisfying as the career they had built? Will economic mobility be the same?

Check out new semantic search on CourtListener which @dominsta.bsky.social and our lab contributed to!

Dominik Stammbach@dominsta.bsky.social

Update: semantic search is now live across all of CourtListener, not just the API 🚀 You can search case law in natural language directly on CourtListener! free.law/2026/05/04/s...

Btw, did a bit of a rebranding of the substack. Will endeavor to post more there. h/t @dbateyko.bsky.social on the Trials & Errors name. Super fitting name for a group whose focus is both in reinforcement learning and in law/governance research. www.trialserrors.ai

trialserrors.aiTrials & Errors | Peter Henderson | SubstackVarious news, thoughts, and findings on the intersection of law, policy, and artificial intelligence. Click to read Trials & Errors, by Peter Henderson, a Substack publication with hundreds of subscri...
Peter Henderson@peterhenderson.bsky.social

This is a challenging legal problem for NeurIPS (and other conference participants)! You might be wondering how this is possible given the First Amendment? I wrote a quick explainer on the current status quo of relevant First Amendment cases & law to get you up to speed. 🔗👇

This is a challenging legal problem for NeurIPS (and other conference participants)! You might be wondering how this is possible given the First Amendment? I wrote a quick explainer on the current status quo of relevant First Amendment cases & law to get you up to speed. 🔗👇

NeurIPS Conference@neuripsconf.bsky.social

NeurIPS is aware of the community's concerns regarding the list of sanctions. NeurIPS is an inclusive community focused on free scientific discourse. We deeply value the research that comes from everyone in our community.

I feel this urgency too. But this is all so utterly avoidable with good policymaking. No one should be left behind because they didn't accumulate capital in 2026. There are so many people who aren't plugged into these conversations or are simply not in a position to do anything about it.

Single parents working three jobs to make ends meet cannot possibly work harder to accumulate capital. They already work hard enough as it is. People in this position should not be "left behind." There should be no "permanent underclass,” as many are worried about.

Even if you're somewhat better off. People also shouldn't have to work themselves to the detriment of their health and families to shield against future labor impacts. They should be able to trust that their government will think ahead and make good policy.

As an aside, my PhD thesis was titled ‘Aligning law, policy, and machine learning for responsible real-world deployments’ for a reason. I think this is a very important area, and I’m excited to see so many excellent researchers working together to move it forward.

The current direction of AI labs is “we’re building something that’s going to replace you and we have no plan to make sure you’re going to land in a better place, but we’ll make billions.” The logical reaction is, “shut it down.” Labs need to get serious on addressing labor impacts.

Many legal scholars talk about lock-in effects for LLMs from the conversation history/memories (akin to social media). But if an LLM can access the info & is capable, you can just ask the LLM to give it to you, making it far easier to switch providers than social media. Good example of that here.

Only a couple of days after my last post, vibe hacking in full force.

Peter Henderson@peterhenderson.bsky.social

Missing from the headline: "using Claude Code." Vibe hacking is already a thing. I've been saying this for a while, but no model-level safeguards will prevent it entirely. What they can do is slow it down enough for us to put societal-level safeguards in place. www.popsci.com/technology/r...

Only a couple of days after my last post, vibe hacking in full force. www.bloomberg.com/news/article...

Peter Henderson@peterhenderson.bsky.social

Missing from the headline: "using Claude Code." Vibe hacking is already a thing. I've been saying this for a while, but no model-level safeguards will prevent it entirely. What they can do is slow it down enough for us to put societal-level safeguards in place. www.popsci.com/technology/r...

As models discover more exploits, we may need something like a responsible disclosure period for major jumps in cyber capabilities. Before release, trusted defenders get privileged access to the more capable model. Together, they scan for vulnerabilities at scale & notify as many as possible.

Unfortunately, the scale of the problem makes it challenging. Even if we freeze at Codex-5.3/Opus-4.6-level capabilities attackers can probably scaffold them to pretty easily identify tons of vulnerabilities.