Post

I at this point am like 90% convinced that reusable/shareable skills packages are a bad thing for LLM users. The 10% is that it can show you what's possible; the 90% is that people want to be incurious and just slap something in the only person who can effectively guide your tasks is you

17 reposts254 likes

Propter use the phrase "build your own power armor" and I think that is right ... but I further think that people MUST build their own, no one elses setup will work right. It doesn't have to be a full harness but it should include some custom interface + skills.mds + project memory ...

1,000% Very few skills are written well enough to be appropriately reusable, and even then only if they closely match your workflows. Bundling them compounds the misfit, default pushing the agent in directions actively antagonistic to your workflows.

I feel like they've ALWAYS been an anti-pattern. They made *some* sense before the models got quite so capable, but now they're bloat / cruft. Just document your APIs and let the agents write their own "skills" if they need to.

This is exactly how I feel about it now. In Jan it was super helpful for me to have skills defined for how to JAX should be used and more esoteric JAX APIs for autodiff behaviors (closure_convert, linear_transpose, etc), but lately, I'm questioning if I need them anymore.

maybe if you're using languages and tools with broader existing knowledge bases this is true. it's pretty important for using our inhouse scripting tools and building in our specific engine. but we aren't sharing them outside our team either (yet)

I’d go further and say that skills in general are deceptive in a not useful way. The number of times I’ve had a limbic crash out at a sparky rock because it didn’t use the skills for the umteenth time while saying it’s using the skill and outputing stuff that clearly proves it didn’t. Arg!

This is particularly glaring when you need output that is precise in a way that statistics just can't do, and the skill documents a non-statistical equation, and it's usage. Then the tool outputs something that is clearly statistical faking of the non-stats math.

I agree for more personal things, eg some generic styling thing. but what about a skill like the reverse engineer anything skill? that’s like directing the agent to follow specific specialist skills github.com/morluto/rea I think these ones which let it do advanced things out of the average

github.comGitHub - morluto/rea: Reverse engineer anything with agents, from app behavior down to native binaries.Reverse engineer anything with agents, from app behavior down to native binaries. - morluto/rea

Old skills are also very overprescribing and work bad with newer models. Also skills written by new models are hilariously bad. You have to remind them "Hey you are writing for YOURSELF, reflect on it, you would have hated this micromanagement" - and only then they are like Ohh!! You're right!