Dave Willner

Dave Willner

@dwillner.bsky.social

Co-Founder at Zentropi. Formerly Head of Trust & Safety at OpenAI, of Community Policy at Airbnb, and of Content Policy Facebook. Strictly cold takes.

10.1K followers2K following375 posts

Pinned

For 17 years working in trust and safety, I've watched talented people burn out on impossible tasks. The problem isn't the people, it's the systems. Traditional moderation requires months of retraining for every policy change. Only big companies can afford it, and even then it works poorly. 🧵 1/9

Dave Willner reposted

We've been using Zentropi's CoPE models on Roundabout, and have had immense success with them. One minor challenge has been the latency of evaluating multiple policies – exciting to see this evolution!

Dave Willner@dwillner.bsky.social

We're shipping CoPE-B-S1, an open-weight decision model tuned for safety use cases. It's a significant step forward for our CoPE family of models in a few ways. A thread: blog.zentropi.ai/meet-cope-b-...

Dave Willner reposted

The newest model in the ROOST Model Community!

Dave Willner@dwillner.bsky.social

We're shipping CoPE-B-S1, an open-weight decision model tuned for safety use cases. It's a significant step forward for our CoPE family of models in a few ways. A thread: blog.zentropi.ai/meet-cope-b-...

When we released the first CoPE almost two years ago, we bet that small, steerable classifiers would become essential AI infrastructure. That bet's held up: CoPE is already the real-time safety layer at Runway ML and some of the largest websites in the world.

So it shines when you need a model to faithfully apply the long, nuanced criteria Trust & Safety policies are built on. It's really, really good at following where you want it to go, no matter how persnickety you need to be.

Third, it's a generally capable open decision model, good at stuff well outside our core domain. It can beat Jev at Tetris (yes really, it's in the blog). A fun party trick, but it also means one safety-tuned model can handle your other classification tasks too.

When every category costs another model call, teams make uncomfortable trade-offs: lumping distinct harms into one blunt classifier, checking only a sample, or quietly leaving some rules unenforced. With CoPE-B-S1 those trade-offs more or less disappear.

Second, it's extremely capable in the safety domain: the most policy-steerable CoPE model we've ever made, and the most steerable model of its (small) size we've measured. It scores 0.766 on our policy steerability benchmark, vs 0.634 for CoPE-B-A4B, our prior text model.

First, it's multi-label. Hand it one piece of content and up to 26 individual policies, and it returns every classification in a single call, at close to the speed of a single policy (controlling for overall prompt length).

That's a pretty big deal for Trust & Safety teams. Real platforms don't enforce one policy, they enforce dozens, against every post, message, and upload, usually in real time. You can get a sense of the high-throughput in this video

Dave Willner reposted

Y’all, I hate to rain on the dead bot parade, but the Yandex data center destruction doesn’t seem to have significantly impacted social media bots. I made temporal graphs of X posts about AfD around the time of the attack and there’s no sign of a sudden dip or residual impact on that conversation.

Dave Willner reposted

wrote on saturday that trump was not going to let democrats avoid the question of impeachment and here we are with an offense that would have had a president impeached, removed and *at least* exiled if it had happened at any point prior to now.

Dave Willner reposted

I’m ok. Make no mistake: this is an attempt to target political enemies and weaponize federal law enforcement. For ten years, Trump has targeted me with baseless lies and smear campaigns. This is just the latest escalation in his campaign of political intimidation. I will not be silenced.

All of this is spot on

Kate Klonick@klonick.bsky.social

This is a very cute clip, but I love it for reasons that the episode doesn't really explore: *All networked technology comes down to chat* Talking to each other is either the foundation of all new networked technology or becomes inexorably tied up with it-- since the early days of the Internet 1/

Dave Willner reposted

New work featured in New Public!: There is no neutral feed algorithm. Each one makes source, ranking, & content diversity choices. @jonathanstray.com & I started MySky so you can see - and change! - these settings. Check it out here on Bluesky, in "Feeds" # newpublic.substack.com/p/meet-the-a...

newpublic.substack.com🎛️📲 Meet the algorithm you can actually control and understandJonathan Stray and Renée DiResta introduce MySky, a feed that shows its work

@masnick.com and I took the time to formalize some thoughts around the unlikelihood of the “sexy apocalypse” as compared to the dumb, disappointing, and far more likely ones.

Mike Masnick@masnick.com

So @dwillner.bsky.social & I took some of our discussion thread from Bluesky last week regarding the "sexy apocalypse" predictions from the EA-aligned "AI Safety" folks, and wrote an article from the perspective (Dave's) of those who do *real* risk mitigation work. The apocalypse will not be sexy.

Dave Willner reposted

"Doomsday isn't when the bridge falls down, it's when it stops being put back up."

Dave Willner@dwillner.bsky.social

Certain kinds of smart people tend to have a "sexy apocalypse" problem when it comes to thinking about risk. E.g. skynet turning everyone in paperclips, while bad, is also *kind of cool and fun* at the same time. Where the reality is that most actual disasters are both stupid and disappointing.

Runway is doing some very cool stuff using CoPE/Zentropi: "We have designed and tested a new fast moderation system that will run synchronously once a user’s input clears moderation.... The synchronous moderation through Zentropi narrows the exposure window to less than half a second."

runway.comRunway News | Moderation in Real TimeRunway built a synchronous moderation system for real-time video generation to scan streaming frames for harmful content in under half a second — layered on Runway's existing safety defenses.

This sort of thing is really only possible when you have a multi-modal model that combines accuracy with speed in the way that CoPE does. Big foundation models can do the classification well, but aren't fast enough. Unspecialized small models are fast enough, but can't do the classification well.

At TrustCon this year I talked about a technique we’ve developed for automatically optimizing content-moderation policies, using an inversion of the binocular labeling approach Zentropi had already pioneered. Today we're shipping the tool that technique became. blog.zentropi.ai/optimizing-o...

blog.zentropi.aiOptimizing our Policy OptimizersToday, we are releasing our next-generation policy refinement tools: policy-only correction, label-only correction, and auto-optimization.
View full thread

On a personal note - figuring out how to get this to work has taken months of focused effort and I am just incredibly excited to get it out the door. Please do try it and let us know how it goes!

At TrustCon this year I talked about a technique we’ve developed for automatically optimizing content-moderation policies, using an inversion of the binocular labeling approach Zentropi had already pioneered. Today we're shipping the tool that technique became. blog.zentropi.ai/optimizing-o...

blog.zentropi.aiOptimizing our Policy OptimizersToday, we are releasing our next-generation policy refinement tools: policy-only correction, label-only correction, and auto-optimization.

To summarize the point - a policy and a set of examples labeled against it aren't independent things you can fix one at a time, because a given label can only be said to be meaningfully right or wrong relative to a specific policy text. So, to improve either you have to work on both simultaneously.

As a result, this new optimizer has three distinct capabilities: policy-only correction rewrites the policy text to better match a labeled dataset. Label-only correction flags and fixes labels that no longer match the policy. Auto-optimization runs both in a loop until the whole stops improving.

Closing them is a loop: run the policy against examples you trust, look where the model disagrees with your reviewers, and let a second agent rewrite the passages behind the disagreement in the policy's own voice. Partners have gone from 65% to over 90% agreement with their own labels. 🧵 4/5

The argument, briefly: once a small model (CoPE) reads a content policy as written and labels against it, editing the policy is editing the classifier, with no retraining cycle in between. 🧵 2/5

This makes the policy's gaps starkly visible: the boundary condition nobody thought through, the term that means one thing on page one and something else on page four. Reviewers paper over those issues without noticing they are doing it. 🧵 3/5

Dave Willner reposted

Wait. They all shake hands at the end? What are we even doing here? He's the US President suing you for $20 billion for reporting on the birthday card he sent his friend the pedophile in which he drew a naked girl & mused about secrets. And they all shake hands?

Dan Diamond@ddiamond.bsky.social

Remarkable camera angle of President Trump as he listens to Wall Street Journal reporters receiving an award for their investigation into Trump’s relationship with Jeffrey Epstein — which they pursued despite Trump’s threats and lawsuit.

Dave Willner reposted

Ben and I are the final act of TrustCon with a live Ctrl-Alt-Speech, today at 4. Joined by special guests @rightsduff.bsky.social & @zoedarme.bsky.social. If you're at TrustCon don't miss it. And if you can track down Ben or I, we have gifts for listeners...

Ben Whitelaw@benwhitelaw.bsky.social

Finally get to record an episode of Ctrl-Alt-Speech in actual person with @masnick.com this week. Real-life audience too (if folks show up). Lots of good stories to chew over. Episode out on Friday — and earlier for Patreon supporters.

View full thread

Because these policies are open source, none of the specific lines are take-it-or-leave-it. If your community needs a different rule, you can fork the text and move the line, and your enforcement follows. 🧵 5/6

View full thread

Platforms have mostly handled this content with one, often blunt, policy because keeping different rules for different ages was operationally out of reach. That's something policy-steerable classifiers change: they let you write the rules per age band, and have enforcement follow the text. 🧵 3/6

The policies Vys built using Zentropi (with a little help on my part) were grounded in developmental research and tested rather than asserted: over 1,100 hand-labeled examples, with F1 scores north of 90 on both age bands. The full methodology is in the write-up. 🧵 4/6

Self-harm policy is some of the hardest work in trust and safety. The same post can be recovery support for one kid and a how-to for another, and what a 17-year-old can contextualize, a 13-year-old often can't. Age isn't a detail in this domain - it sits at the heart of the problem. 🧵 2/6

Dave Willner reposted

Today we beat Stephen Miller’s lawsuit mill in court! For 3 years I’ve been a defendant in a case filed by America First Legal on behalf of Gateway Pundit & a random antivaxxer we’d never heard of until she sued us. The plaintiffs claimed we “censored” them. It was a lie. They just lost.

Dave Willner reposted

Two years ago, my colleagues and I were sued by Stephen Miller's America First Legal. The Hines v Stamos case — built atop a number of false allegations — was a central element of the "censorship industrial complex" myth. Today, that case was dismissed (by a Trump judge in LA). We won.

Dave Willner reposted

Was Hines v Stamos Cheryl Hines suing John Stamos over the long-awaited Full House/Curb Your Enthusiasm crossover? Nope, it was an attempt to suppress academic research into the organized manipulation of online platforms. After three years we finally won. But at great cost.

Kate Starbird@katestarbird.bsky.social

Two years ago, my colleagues and I were sued by Stephen Miller's America First Legal. The Hines v Stamos case — built atop a number of false allegations — was a central element of the "censorship industrial complex" myth. Today, that case was dismissed (by a Trump judge in LA). We won.

Dave Willner reposted

Replying to @qjurecic.bsky.social

2. for that reason, I will be interested to see how the court reform movement on the left processes this ruling. 5-4 is a lot harder to sell as a stable victory. If anything this should make the klaxons blare louder