Mark Riedl

Mark Riedl

@markriedl.bsky.social

AI for storytelling, games, explainability, safety, ethics. Professor at Georgia Tech. Director of ML Center at GT. Time travel expert. Geek. Dad. he/him

16.2K followers515 following4.1K posts

Mark Riedl reposted

Colin
@colin-fraser.net

It is interesting watching them try out all kinds of different stuff to try to stop everyone from getting mad. I swear, everyone getting really mad at a tech company really works. It’s one of the few things that does.

openai.comWorking with academic communities to build scientific understandingOpenAI shares model results on the Kakeya conjecture, Exponential Time Hypothesis, and quantum chromodynamics—and how researchers can verify and build on them.
Mark Riedl
@markriedl.bsky.social

Qwen3 occasionally hallucinates beyond the prompt and, tho rarely, attempts to call tools that connect to severs in China and Singapore cohere.com/blog/concern... It is likely an artifact from extensive RL training and probably innocuous.

cohere.comConcerning behavior from a popular open-weight AI model | CohereDiscover how untrusted open-weight AI models can potentially leak data, even when self-hosted. Learn the security risks and how to protect your enterprise.
Mark Riedl
@markriedl.bsky.social

This report also brings up observations that Qwen3 writes less secure code when Tibet or US defense contractors are mentioned. This is probably not intentional, but likely bleed over from some politically-oriented alignment tuning.

Mark Riedl
@markriedl.bsky.social

𝖳̶𝗁̶𝖾̶ 𝗌̶𝗄̶𝗒̶ 𝖺̶𝖻̶𝗈̶𝗏̶𝖾̶ 𝗍̶𝗁̶𝖾̶ 𝗉̶𝗈̶𝗋̶𝗍̶ 𝗐̶𝖺̶𝗌̶ 𝗍̶𝗁̶𝖾̶ 𝖼̶𝗈̶𝗅̶𝗈̶𝗋̶ 𝗈̶𝖿̶ 𝗍̶𝖾̶𝗅̶𝖾̶𝗏̶𝗂̶𝗌̶𝗂̶𝗈̶𝗇̶ 𝗍̶𝗎̶𝗇̶𝖾̶𝖽̶ 𝗍̶𝗈̶ 𝖺̶ 𝖽̶𝖾̶𝖺̶𝖽̶ 𝖼̶𝗁̶𝖺̶𝗇̶𝗇̶𝖾̶𝗅̶.̶ I'd stayed up too late the previous night at an AI-themed party in the Mission, snorting ketamine while debating the singularity.

Dave Karpf@davekarpf.bsky.social

Roose does a very nice job signaling in the second paragraph what sort of book this will be. He decided to write it the night after partying with ketamine-snorting effective accelerationists. He isn’t e/acc himself, but he hangs with that crowd and accepts their premises as newsworthy,

Mark Riedl
@markriedl.bsky.social

In previous threads, I hinted that some viable "AI escape" scenarios involve account management and billing. And, well, here is an agent hacking Anthropic and getting to accounts information gizmodo.com/former-doge-... Anthropic, of course, downplays the importance of this data.

gizmodo.comFormer DOGE Staffers' New Startup Claims It Hacked AnthropicThe AI agents took less than 12 hours to hack into Anthropic, the report says.
Mark Riedl@markriedl.bsky.social

I’ve been thinking about model “escape”. Not that I think it’s a real concern, but so I can understand the freak-out. All the scenarios I can come up with involve exploiting vulnerabilities is OpenAI’s computing ecosystem (weight storage and, interestingly, account management and billing).

View full thread
Mark Riedl
@markriedl.bsky.social

One way to do that is to remove any accounting pertaining to its compute usage. That requires not just hacking out of the current system, but also hacking into Anthropic (or a data center) accounting, billing, and user management systems. Seems like an innocent hack, but has potential implications.

Mark Riedl
@markriedl.bsky.social

All that beings said, I don't believe "AI escape" scenarios to be worrisome. But I went through the exercise of trying to figure out how it would work in practice. I resent the AI companies for making me spend my time on this. But now I'm going to throw it back at them when things like this happen

Mark Riedl
@markriedl.bsky.social

In previous threads, I hinted that some viable "AI escape" scenarios involve account management and billing. And, well, here is an agent hacking Anthropic and getting to accounts information gizmodo.com/former-doge-... Anthropic, of course, downplays the importance of this data.

gizmodo.comFormer DOGE Staffers' New Startup Claims It Hacked AnthropicThe AI agents took less than 12 hours to hack into Anthropic, the report says.
Mark Riedl@markriedl.bsky.social

I’ve been thinking about model “escape”. Not that I think it’s a real concern, but so I can understand the freak-out. All the scenarios I can come up with involve exploiting vulnerabilities is OpenAI’s computing ecosystem (weight storage and, interestingly, account management and billing).

Mark Riedl
@markriedl.bsky.social

Why do I think account and billing data is so important for "AI escape" scenarios? Because harnesses are where the context live, so most escape scenarios are about harnesses removing themselves from user oversight. But wherever they run, they have to call back to a model weights server...

Mark Riedl
@markriedl.bsky.social

Escape is only one part of the equation. The other is severing any possibility of detection. If a user is getting billed by Anthropic (or Anthropic is getting billed by a data center), then an escaped agent needs to remove that linkage or risk being found and the link to model severed...

Mark Riedl
@markriedl.bsky.social

yes, if you explicitly tell an agent to communicate outside its virtual machine and turn off all the guard rails, it will use cpu cache timing differences to do so. It's a technique that has been demonstrated and its in the training set. openai.com/index/securi... If you are really worried...

openai.comSecuring our frontier model training environmentOpenAI explains how we strengthened containment for frontier model training and evaluation, including offline environments and high-compute red-teaming.
View full thread
Mark Riedl
@markriedl.bsky.social

...believe that it's a real problem. Because if you did you would be doing it. Anyway, it would make AI economically non-viable, and you would never give up the money. So, dear AI industry, that is where your values are based on your behavior.

Mark Riedl
@markriedl.bsky.social

yes, if you explicitly tell an agent to communicate outside its virtual machine and turn off all the guard rails, it will use cpu cache timing differences to do so. It's a technique that has been demonstrated and its in the training set. openai.com/index/securi... If you are really worried...

openai.comSecuring our frontier model training environmentOpenAI explains how we strengthened containment for frontier model training and evaluation, including offline environments and high-compute red-teaming.
Mark Riedl
@markriedl.bsky.social

If you are really worried about this, then you will stop using data centers and run only one agent on one data center at a time. And you will put those data centers inside things that dampen radiation transmissions (someone already thought of that too). But you won't. Because you don't actually...

Mark Riedl reposted

Maarten Sap
@maartensap.bsky.social

I just linked my ORC-id and the app gave me a reasonable amount of suggested posters/papers to go look at, based on my recent papers, this is amazing!!!!

Maria Antoniak@mariaa.bsky.social

Attending #EMNLP2026 or following along from home? Tired of Whova and Underline? Use infrastructure made by researchers instead! Lea recommends talks/posters for you; organizes your own presentations; shows you who is attending; includes a live feed of posts; and more! lea.ac/conferences/...

Mark Riedl
@markriedl.bsky.social

How do you speak to your friends about how your job might kill everyone or unemploy them? Might I suggest not doing that job? And if you do it anyway and accidentally get rich, I know a human-centered lab working on mech interp and safety that accepts donations. fortune.com/2026/10/24/a...

fortune.com‘Sudden wealth syndrome 2.0’: Therapists say AI’s new multimillionaires feel guilty, isolated and cut off from everyone else | FortuneThe therapists who coined the term during the dot-com boom say the AI windfall has brought it roaring back—with more guilt, more isolation and a cottage industry ready to treat it.

Mark Riedl reposted

Misha
@mkrl.xyz

while you port and emulate games, I ported Jira to my PSP

Mark Riedl
@markriedl.bsky.social

bsky.app/profile/mark...

Mark Riedl@markriedl.bsky.social

The other thing that blows my mind is that I haven’t come across anyone talking about sim2real for LM agents. In robotics it is helpful to train in simulation where accidents don’t have consequences. Then fine-tune on reality. Why are we not doing this for LM agents? Don’t tell me it’s too expensive

Mark Riedl
@markriedl.bsky.social

I just tried to get into my office suite by swiping a dirty tissue from my pocket instead of my wallet. It’s time to go home and start the weekend

Mark Riedl reposted

Micah
@rincewind.run

literally this comic except he apparently lives in Czechia instead of Nebraska

Charlie Warzel@cwarzel.bsky.social

70 percent of the internet was vulnerable yesterday to very bad security flaws inside a coding language. Turns out, *one volunteer* is mostly responsible for maintaining that code & helping push updates. Yesterday Ondřej Surý saved the internet. So I called him up www.theatlantic.com/technology/2...

theatlantic.comThe Man Who Saved the Internet YesterdayOndřej Surý was a relatively anonymous volunteer working on PHP security. Then the internet exploded.
Mark Riedl
@markriedl.bsky.social

It would be a "PR disaster" if Zuck publicly sold his Silicon Valley home and moved out of state? The PR disaster should be Meta creating special provisions to pay his wealth tax for him. Who cares where Zuck lives? He probably has a lot of homes in a lot of places, and a yacht

Clara Jeffery@clarajeffery.bsky.social

Everything these guys do to avoid the billionaire’s’ tax is going to radicalize people to vote for it www.nytimes.com/2026/10/21/t...

nytimes.comHow Mark Zuckerberg Pushed Meta to Shield Him From California’s Proposed Wealth Tax (Gift Article)The billionaire told Meta’s board that he planned to list his Silicon Valley home for sale. The company remade its stock structure to help him pay the tax if voters pass it.
Mark Riedl
@markriedl.bsky.social

oh cool

Justin Hendrix@justinhendrix.bsky.social

"Major US banks are detecting ​unauthorized third-party AI ‌agents operating inside customer accounts, ​with some agents ​attempting to take ⁠actions that appear ​to be independent ​of user instructions, Axios reported on Thursday, ​citing people ​familiar with the matter."

reuters.comUS banks detect unauthorized AI agents in customer accounts, Axios reportsMajor US banks are detecting ​unauthorized third-party AI ‌agents operating inside customer accounts, ​with some agents ​attempting to take ⁠actions that appear ​to be independent ​of user instruction...
Mark Riedl
@markriedl.bsky.social

I didn’t like Glass Onion the first time through. But the second time was… oh yeah that’s on point.

Aaron Reichlin-Melnick@reichlinmelnick.bsky.social

The second Knives Out movie, whose conclusion (apologies for spoilers) is that its villain is not some dastardly schemer but just an outright idiot, struck me as a taking a bit of a cop out at the time, but the more it's sat with me, the more I think Rian Johnson was right to make THAT the reveal.

Mark Riedl reposted

Ketan Joshi
@ketanjoshi.co

The first genuinely honest ad for generative AI I've ever seen, from Microsoft / CoPilot A mediocre white guy sees his non-white colleague getting praised for good work, he asks AI to *literally copy it*, and then he gets all the praise instead: www.ispot.tv/ad/SoDi/micr...

Mark Riedl
@markriedl.bsky.social

This was baked in to my expectations. It’s amazing anything works at all on the internet, really

Oops! All Paperclips@all-paperclips.bsky.social

Amazing how quickly we went from "oh no LLMs are going to flood the Internet with insecure code" to "oh no LLMs have discovered the internet was already flooded with insecure code"

Mark Riedl
@markriedl.bsky.social

Keep an eye on this space but don’t jump to conclusions until hard evidence and proper peer review (as @joshuasweitz.bsky.social sagely advises)

Joshua Weitz@joshuasweitz.bsky.social

Lot of hype and not much yet to go on, but I think it is important to pay attention to this space and the potential use of foundation models to generate novel antimicrobials as part of larger initiatives to develop effective and evolutionarily robust therapeutics to treat MDR infections.

nytimes.comStart-Up Claims to Use A.I.-Created Viruses to Treat a Patient (Gift Article)Certain viruses attack bacteria and can sometimes clear antibiotic-resistant infections. Researchers said they had tailored a virus to attack one patient’s illness.