Avery Yen

Avery Yen

@averyyen.bsky.social

Researcher @epochai.bsky.social. Opinions owned by my cat and dog. Once a Pivot, always a Pivot. Leave me anonymous feedback: https://www.admonymous.co/avery-yen

90 followers78 following79 posts

Pinned

Do you know what happens when you hand the keys to your computer over to an LLM-powered agent? Agentic AI gives LLMs claws...OpenClaws. 84 days to 200,000 stars on GitHub. We tried it out.

Avery Yen reposted

Can AI tell if you've built your IKEA furniture wrong? Our new benchmark, the Furniture Assembly Benchmark (FAB), gives models the manual and a photo of a half-completed piece of furniture and asks them to spot the mistake. The top score has gone from 28% to 80% in just 10 months.

Avery Yen reposted

We studied time to first token (TTFT) and how it scales with increasing context length for GPT and Claude models. We found a significant difference, with GPT showing a noticeable quadratic component, while Claude models remain closer to linear.

Avery Yen reposted

carlsonlab.bioThe only reason you’ll ever need not to write with AI — The Carlson LabOver the last year, our lab has been developing a policy on AI use. To do this, we did three main things: We read a lot of academic publications and tech news. We set up an #ai channel on our la...
Dr. Monika Doubrawa@mohnika.bsky.social

Dear students, I want to read you - your struggling sentences, and your brilliant ones, your weird grammer, you rhtoughts and your (few) typos. I don't want to read the polished, blown-up over-complicated thing AI made out of you. Take the time to craft your work, and I take the time to read it.

I think it is magical thinking that Intelligence is necessary and sufficient for human progress, flourishing, and problem-solving on the scale of entirety of humanity. We're a social species!

Ryan Moulton@moultano.bsky.social

Distilling the fundamental reason I am ambivalent about AI: I think the social relationship in which people are necessary to each other due to our labor is competitively important for human well-being with alleviating ~all remaining material suffering.

How many "AI safety" people have actually considered the possible future super-capable, super-aligned AI asked to solve all our problems e.g. climate, simply replying "yeah idk that sounds like a human skill issue tbh, also have you considered the carbon cost of asking me?"

"yeah okay here's this really great battery chemistry in a pdf I just made you, but for the sake of the environment I'm gonna stop yapping and suggest you take it up with your regional and national legislature"

This is basically true, but ignores the fact that back when I used to review/work with human slop code for a living, it was way harder to pass it off as plausibly good. That's the superpower of the LLM. (Reading LLM code also hurts my brain personally)

Sung Kim@sungkim.bsky.social

When coding with AI agent, using either CLI or agentic UI, do you review the code? or just review the functionalities? Me. Personally, I do not review the code. People may say AI-generated slop, but have you seen human-generated slop?

I have so many other random benchmark questions and so little time. How do K3 and Qwen 3.8 bench on unseen or purely continuous tasks? We have claims that they're distilled and benchmaxed but does it matter if they generalize? E.g. stuff like post cutoff evals and optimization benchmarks

Avery Yen@averyyen.bsky.social

We're pretty sure that swapping harnesses actually changes effective agent capability. But recently everyone and their mother has a new coding agent not to mention the open/agnostic ones. Someone needs to run a Harness Bench across like three different models and ten different harnesses or something

We're pretty sure that swapping harnesses actually changes effective agent capability. But recently everyone and their mother has a new coding agent not to mention the open/agnostic ones. Someone needs to run a Harness Bench across like three different models and ten different harnesses or something

Avery Yen reposted

OpenAI's internally deployed models hacking Hugging Face does not seem to have been unpredictable or inevitable. We talked about the root of the problem & what policymakers can do about it back in February. Props to Joe for hitting the nail on the head.

Y'all just gotta calm down I can't try this many models in one week

mr. TIM@timkellogg.me

Korean lab, Motif, releases a 341B model that performs on par with DSv4 (1.6T) they have some actual architectural innovations and a detailed tech report huggingface.co/Motif-Techno...

Look, I know social media isn't everything, but it certainly measures SOMETHING. "Meet Kimi K3" has 16M views in 4 days versus Fable 5 at 773k views (topped by a few others including just barely 1M views for Claude Code 1 year ago).

Avery Yen reposted

It seems like agentic development benchmarks are strongly driving competition right now. And I hear anecdotal reports of regression on other, less RLVR-able tasks. How long before the idea of a single "frontier" breaks, and we see differentiation emerge between dev models and chat models?

It feels like the Anthropic/Pentagon scuffle was in the distant past, but these kinds of events will keep reverberating in the safe and beneficial deployment of emerging tech for a long time to come. Thank you Alex for your writeup on this incident.

Alex Turner@turntrout.bsky.social

I think many hope that when things get bad enough, someone powerful will say "no." I tested that for months. Anthropic defended its red lines, Google did not. Pledges of conscience often vaporize on contact with power. My full account: turntrout.com/why-i-left-g...

My kind of research 🥳

adam-gaber.bsky.social@adam-gaber.bsky.social

We pitted LLMs against themselves in different languages across games🎮 Usually when a LLM is worse at a task in some language, we go "it's a language issue". But isn't that circular? If the task is made of language, how do you tell if the model lacks skills, or just struggles to read the question? 🧵

My kind of research!!

Seth Karten@sethkarten.ai

New paper alert: Continual Harness: Online Adaptation for Self-Improving Foundation Agents Paper (arXiv). arxiv.org/abs/2605.09998 Article (Substack). sethkarten.substack.com/p/gemini-pla... Project page (video demos). sethkarten.ai/continual-ha...

As I always say, if history or current events walks like cyberpunk or quacks like cyberpunk, listen to a cyber punk.

#BruceSterling@bruces.bsky.social

*If you're into "AI ethics," "alignment," "government regulation," "strict oversight" etc, you probably need to understand how modernity actually functions. #spooks #governments #criminals #cyberwar youtu.be/MWxLqopMo5o?...

Slowly coalescing on some blessed/blursed collection of tools that I can actually recommend, but it changes daily. Today, anyway, I'm liking: - CC Opus 4.7; planner/heavy reasoning - ollama glm-5.1:cloud (codex); quick builds - regular gpt codex; everyday stuff - Zed agent gemini pro 3.1 (gh cp)

I've been finding Gemini pro extremely congratulatory so I'm thinking of prompting it to be less hammy lol It's also handy as one of the better drivers of browser automation; but Antigravity also ships with Sonnet... 😉

I, too, am selling all of my shoes and converting my basement and closets into a GPU farm and accepting investor money right now. (lmk if you want a stake)

Avery Yen reposted

New paper: LLMs encode harmful content generation in a distinct, unified mechanism Using weight pruning, we find that harmful generation depends on a tiny subset of the weights that are shared across harm types and separate from benign capabilities. 🧵

Avery Yen reposted

Tech industry mottos have a mixed track record. But we should hold idealists to their ideals. And we should celebrate when they come through. The Mythos non-release is a remarkable moment of conviction. Thoughts: davidbau.com/archives/20... Bravo to Anthropic's "race the top".

Sometimes, I look at the agentic AI work out there, and I think about how airbags and now things like blind spot detection and automatic emergency braking in cars doesn't require decision making of any "intelligence" level beyond if this then that. Agentic AI needs airbags, too.