Post

Using agent tools and learning them has been so much fun! Here's a personal tool I started 3 weeks ago and deployed last night: github.com/psumiya/option-insights I coupled Spec Driven Dev w/ Vibing for specific tasks using #Kiro + Claude 4.5. Sessions paired with Grimm episodes post-bedtime.

Quick prompting and incomplete planning has almost always been the underlying issue when I see these things fail. Sonnet 4.5 alone is good enough for 90% of coding requirements, IMO. The last 10% requires a top model like Opus 4.5 + careful, knowledgeable human expert + multi-step planning.

including a note on cost declining in particular

Yeah that was a bold move of you! I actually wanted to quote it and write my thoughts but she's a huge account and it feels like being a trans person that finds utility in LLMs makes you a class traitor or something.

The Claude Max subscriptions are incredible value. I’m currently on Droids 200$ plan, I am unsure how far into the month I’ll get before I run out; but I reckon I’ll get quite far!

I am surprised people still try to deny this. in a few niche fields/applications it can be not great (as karpathy has mentioned; he likes it for web apps, not so much building language models) but overall it is obviously a huge productivity boost for most programmers willing to use it properly

Armin's post that my reply was attached to got retweeted by some VERY high traffic accounts which exposed it to thousands of people from different bubbles tools.simonwillison.net/bluesky-quot...

Yeah. Its wild how different sonnet 4 and 4.5 are. I was a skeptic til i tried 4.5 and made a copilot instructions file and then was off to the races

The moment it gets close to "architecture" it immediately falls apart tho, duplication galore, fudged up class hierarchy, the more the merrier

Yep. The new Opus is really quite something too.

To the extent possible, try to turn your problems into software shaped problems

on a small scale, yes, they generally produce good code, but the more agent tools you use, the further you lean towards vibe coding, which still produces terrible code *bases*. the architecture is generally quite bad and they tend to redundantly reimplement the same thing in many different ways

i would / do however, it's been a process to get understand how to do so safely / usefully, and one i'm not 100% confident i could explain to others yet

I just started playing with PyCharm and noticed it guessing and suggesting code. Trying to find out why people are losing their shit over Python. I'm thinking it's really the libraries rather than the language per se.

Kicking the hornet's nest. Those are fighting words for some...

You'll find that copyright & IP issues are an unexpectedly major obstacle to general corporate adoption, as content generated by LLMs can't be copyrighted, while inventions created by LLMs can't be patented. To be sure, the law will change, but it hasn't yet.

Also side tools that can be super vital but don't go into production The models just chew through visualizations They are also amazing at "repair", "update", or "sort" tools that help manage things behind the scenes

This only works well for experienced devs who can spot the 20-30% of defective outputs that the models produce. There’s also substantial difference between code quality in different languages. It’s best at greenfield python. Ask it to convert C into Java & the results are much less obviously good.

... especially if you embrace BDUF and other old waterfall methods, in my experience at least. Quick way to "working software as the spec".

If this is true why is the Datasette v1.0 release taking so long ;-)

Thanks for doing this, Simon. And kudos on helping to maintain a civilized conversation around this. I had opus 4.5 generate an svg based dancing turkey the night of TG - thought about your pelican on a bike eval as I was doing that

Stop. You’re embarrassing yourself. I am embarrassed for you. anthonymoser.github.io/writing/ai/h...

I had a problem where an LLM helped me port some R code to C++ (i dont know C++) and got a 15x speedup. I was super careful about reviewing and comprehending the code. that experience, like, makes the benefits pretty undeniable to me.

They can definitely produce a ton of slop and require handholding. As bizarre as it sounds, there is a skill to using them.

two steps forward, one step back. when I craft a "PR" in collaboration with an agent, I more often than not usually end up crystallizing that work down into a single commit.

Tooting my corporate horn here, but I’ve found it helpful to use Sonar’s complexity analysis to keep the LLM in check.

My experience is that they're quite shit for c++. Test cannot really confirm that you don't get UB in all circumstances due to threading issues so it's difficult to confirm it hasn't created you something that only definitely works for the test cases

Not any more! bsky.app/profile/simo...

That’s not a question an LLM will have a good answer to, even just simply because of the training cutoff date

Google's AI overviews are still the worst widely available AI models! I just checked and it's STILL saying Encanto 2 is a real movie, that bug is nearly a year old now simonwillison.net/2024/Dec/29/...

Google‘s search LLM is the village idiot of current models.

In theory, I can envision a "code-understanding" system that could prevent race conditions (which are notoriously hard for humans to see). But I'm skeptical we'll see that delivered, in reliable form, any time soon.

I can see how 100000000 working react programs on GitHub can guide LLMs to generate another one, but maintaining all the highly implicit invariants in something like RocksDB when adding an enhancement ? Different worlds.

This code is hard to write! That's why I want help!

it works well in specific in my experience and it seems to be getting better in the general, though it's hard to separate user skill, model capacity, and framework improvements in that (my first vibecodes were over-complex trash, but that was probably my fault as much or more than claude's)

Those that disagree have not fully dove in yet. It's been transformational for me.

I was hoping this thread wouldn't end with another tiny vibe coded tool I'm starting to see the results of people being encouraged to use LLMs, being given complex tickets in established projects, with tech where they clearly don't know how to tell good code/design from bad, and it's not pretty

It’s just true. It has been an enormously useful tool in my experience.

Does Nat know you're a masochist

and finally, you reject the possibility of other ways to fulfill the same "productivity" you mention, because you simply haven't defined "productivity", and you don't know or understand the field of tooling that others have worked in for decades

It was also meant as commentary on how absurd it is that saying "LLMs are good at writing code" really does have the potential to kick off a huge flamewar in November of 2025

That's an interesting point. I'm in a 4 person company so, by default, we tend to work in silos so all of my experience (which is good) has been in single-person projects

i'll echo other advice here: ai is more useful for exploration when you're just beginning ask questions about existing codebases and have it lead you to the evidence be ready to disagree with it socratic dialogue (aka "just asking questions") goes a long way to making sure it's not full of shit

Don't apply AI to boost productivity. Treat it as a learning resource: an infinitely patient buddy willing to explain everything, sometimes getting it wrong. Don't ask for solutions. Ask for options & rationale. Think through what it's saying, challenge it if sounds fishy, drill down. Be curious!

As someone probably in between yourself and the OP, I'd say use it as a learning tool. Treat it as a paired programmer and ask questions. If you don't know what you're doing and blindly let AI run wild, you'll not recognise if it's going off track, or potentially doing something extremely flawed.

It's been so long since I was a beginner that I can't really give credible advice here My guess is that the very fact that you're worried about this puts you in a great position to solve it yourself through thoughtful application of the technology (I use it as a learning tool myself all the time)

Which, I think llms fit into very well for things like jargon discovery presumably?

its kinda bonkers how many old ideas for software artworks have suddenly become frameworks and mockups of usable software ideas that exceeded my knowledge/skills and could never justify paying someone else to build are becoming real like an automated mark lombardi network narrative generator

The Bluesky dogma is always the same things: - I read it ate all the water and pooped fire (and this is probably true about xAI) - Understanding a topic requires knowing it to the atom. If you use it then it makes you more dumberer - Elon Musk pays me to make you avoid legitimate tech developments

Yea but that was true in 2021 as well. The instruct models were useful.

"AI now solves the exact same problem StackOverflow used to solve—just faster, friendlier, and without the top comment asking why you’re writing bad code in the first place." ouch, but no lies detected

Skippy the Intern: A great labor-saving tool, but you gotta know how to check the work. (To me, 'Skippy' is a completely non-gendered name.)

Especially because the right answer is totally subjective and depends so much on the type of work, and an individual’s personal technique and style.

I'm absolutely fine with developers knowingly opting out of AI assistance for ethical reasons - and being willing to accept any career implications that might have as the price of staying true to their principles. I admire that! But I want them to have accurate information

I have seen them coming around. They see enough colleagues happily using the new tools and eventually giving them another try.

I’ve been using cursor for quite awhile and I’m easily 10x productive with it. Just swapped to Claude code and I’m interested in all the advice you might have

More data. pivot-to-ai.com/2025/08/31/3...

Interesting how one third said fixing the slop took longer than not using the AI while the other 2/3 JUST ACCEPTED THE SLOP.

lol

This week I built 3 prototypes for a feature, which add 2 new capabilities to a complex system, touching numerous parts of 1m+ LOC repo. Each time I tried a different architecture, improving on the last. I estimate each would have taken > week without an LLM. But I did all 3 in a week!

Yeah I had tried a few times. And had very mixed success even in self contained tasks. Now I've managed to wrangle it for valuable product work, versus say the odd bash script or custom ESLint rule (they are valuable too! But they're just ancillary tasks)

They are glorified template engines. If you know what you're looking for, you're going to help them to go faster, but as soon as you're doing anything complex, they are going astray quite fast

"who take the time to learn how to effectively apply them" is the key 🥲

By only using LLM one risk to only be as good as the tool. Which to me feels like a liability on the job market

what for ? If I code as well as the llm, why need to use it

No issue with using LLMs/AI like this - it's a great code writer and coding fault finder. Also to produce indices and find information across vast sources. Doesn't make it right to take and use copyrighted content for commercial gain without paying for it.

This is a problem because of the old adage: "Debugging is twice as hard as writing the code in the first place." Complex solutions have a much higher maintenance cost which will limit your ability to support the code base. AI just supercharges the ability to generate tech-debt if... you allow it.

claude definitely follows the "make it work, then make it work well" aesthetic you just have to be ready to do the second part, instead of accepting "it's alive" as the stopping point which is true of most code i write too, so it wasn't a real hard transition once i recognized it

only some of, though, because i worked crappy web dev the first half of my career unlike the nephew of the owner of the tv station, i can ask an llm to fix its hideous mistakes and it will, often the first time that nephew i essentially had to follow around with a pooper-scooper (and i quit asap)

I am genuinely fascinated by those events. I couldn’t have predicted them at all. I can’t quite get my head round where the passionate opposition comes from

This is a toy example and it has not produced excellent code, so it doesn't seem like very strong evidence for your claim.

Yup. Never before has so little been done with so much.

Yeah, they are missing a trick there I always ask for my artifacts as HTML with inline JavaScript and no React because then I can copy and paste them out and host them elsewhere really easily

bsky.app/profile/cait...

Funny, but no