David Manheim

I'm not saying that the problems AI companies are having with RL environments and reward hacking are just evaluation design problems that they never bothered solving, but...

In light of the recent results... I'm kind of hoping for a proof that P=NP for polynomial-time decision algorithms, but with a construction lower bounded by some ridiculous 3- or 4-digit degree polynomial making it irrelevant, so theoretical CS people scream about it and it doesn't matter.

I'm not saying that the problems AI companies are having with RL environments and reward hacking are just evaluation design problems that they never bothered solving, but...

Excited to be here in Montreal this week for the @isostandards.bsky.social plenary on Artificial Intelligence standards as an expert on behalf of the Standards Institute of Israel.

#ChemicalWeapons #BiologicalWeapons #ArtificialIntelligence #Terrorism A Pre-mortem on The First Major AI-Enabled Terrorism Event and The Choices That Could Enable It cutt.ly/GyR6g6gC [PDF File] Abstract Frontier AI developers’ model reports and the dedicated risk-assessment teams at ...

🚨New paper (possibly the most important one I have written yet). We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios. Led by @kamachee.bsky.social

"Propensity to evade monitoring" - ? It sounds like AI companies are insisting that their models are just tools, to evade responsibility for creating something legitimately anthropomorphic that has goals the companies don't want.

"Be the out of distribution event you wish to see in the world" is a darn good slogan.

New post arguing that while agreeing on what to do about loss of control is very hard, agreeing that everyone, **including the future AI systems themselves**, should enforce agreements is a good move.

Fantastic story from Stefan Larsson @lund-university.bsky.social giving the keynote today at #AIES2026 He thought he was done working on copyright law, and had moved on to AI issues - then found out that *his book on copyright* had been pirated to train LLMs via the Anthropic class-action lawsuit.

Hear, hear! But this is not just an issue created by AI use; the current role of reviews is that all criticism must be a shortcoming of the paper, because the only way to interact to improve a paper is introducing veto-points. (And this is not how "peer-review" worked before journals!)

Heading to Malmo for #AIES2026, Sun-Thurs. Looking forward to seeing people there, feel free to DM / email / etc. if you'd like to meet up!

Yes - if there was ever an excuse for not preregistering because of the extra work, it's certainly gone now.

There's something especially poetic about using precisely manufactured ultra-pure silicon wafers (GPUs) to compose and make a music video about methods for keeping high purity and remove impurities during materials engineering. www.youtube.com/watch?v=8Pnl...

in case you didn't know, deck.blue is currently my main source of income while i'm also getting into photography work here in são paulo, so i really appreciate everyone that's currently subscribed to the patreon; making all this possible if you can't subscribe but want to help, there's a ko-fi too!

“The Trump-installed board that runs the Kennedy Center told a federal judge the building will have to be torn down unless President Donald Trump's name goes on it.“ Even Hitler would have blushed at such an insane demand, but it’s totally normal behavior for Needy Amin.

I think there are some interesting question at the intersection of LLM personas and agent swarms, and I haven't seen anyone explain this clearly; here is my attempt. davidmanheim.substack.com/p/concerns-a...

Public opposition to data centers somewhat matches the vibes of the people worried about AI safety, but actually stopping their construction isn't meaningfully helpful for AI safety, it's at best irrelevant.

@pkrugman.bsky.social Since you're evidently not on twitter:

Yes!

I was really happy to appear on the @futureoflife.org podcast this week to talk about @evals-consensus.ai and the challenges of evaluations of AI systems, along with discussions of Goodhart's law, AIxBiosecurity, AI persuasion, forecasting, and human oversight of AI.

Over on the bird site, Anthropic employees are using their slop machine that 'can't think' to solve longstanding mathematical open questions which have eluded humans for close to a century.

I keep seeing claims AI companies will get rich, or won't make any money, or that open source models will be cheaper and undercut prices, and similar claims that ignore the simple fact that the economic pricing power is all about compute and token generation costs. davidmanheim.com/AI-Economics/

"Technologies that have first order impacts on coordination and production, or that empower groups in other ways, tend to differentially benefit the powerful in ways that are harmful to others, either directly or indirectly"

New post: Huge technological revolutions aren't usually positive for those living through them - and this bodes poorly for AI even if it is a normal technology. davidmanheim.substack.com/p/if-ai-is-n...

GPT-5.5 seems to do more 'hacking per dollar' ...Mythos does more 'hacking per token'" - @peterwildeford.bsky.social Thinking about risks from bad actor's access to scaled models, the costs involved here are still absolutely tiny compared to other attack modes.

It's hard to avoid the conclusion that Bluesky has been a net negative for US politics. They corralled everyone on the left into a little glass fishbowl where they shout at one another & everyone else ignores them. Meanwhile, all the pols & institutions stayed on X & are being dragged farther right.

The cognitive costs of choosing between different LLM versions or subagents for different tasks, picking or managing the thinking levels, whether to use /fast, and managing token budgets is an amazing illustration of why firms bundling labor into employee salaries is so much more efficient.

I think it's entirely appropriate that the people calling AI a tool are also people who would be insulted if we called them a bunch of tools.

Just sitting here waiting for the @andymasley.bsky.social haters reactions to his newest post...

@ioda.live Question: Is February 2021 Texas historical data available? (We keep getting empty responses for historical US bgp data from the API.)

The proportion of Philosophy articles I'm asked to review that have undeclared LLM writing is too damn high! (I think LLM usage for writing is often fine, even / especially if the writing itself is about LLM's ability to reason. But it's supposed to be disclosed, so disclose it!)

"Chinese companies cannot legally fire employees simply to replace them with cost-saving artificial intelligence, courts in the country have ruled, setting a significant precedent for labor rights as automation sweeps the tech sector." There goes the "China is trying to win the AI race" narrative.

We’re raising funds to print a brand new book of compiled SMBC comics on the topic of parenting, alongside our preorder for Sawyer Lee! Check out the project here : www.kickstarter.com/projects/wei...

It's not just in the prompt, it's there twice! (Mistake, or necessary overkill? Who knows!) bsky.app/profile/emol... github.com/openai/codex...

"Draw me a highly detailed where’s Waldo image with people or items to find, but of an ISO SC42 standards conference. Make sure to make it funny with in-crowd jokes."

One problem with making predictions in public is that when I say I'm 75% sure of something, and someone else responds that they are 99% sure I'm wrong, they are using numbers as rhetoric, and I'm trying to make sure that I'd be right approximately 3 out of 4 times.

This is the whole alignment problem. All of it, encapsulated. Low-probability behaviors by the model, just make your training environment=your test environment, "what's a capability vs. a propensity", non-adversarial generalization, Goodhart's law. The whole damn thing