Post

Smaller models forget system prompt instructions. What if we attach instructions to specific tools and deliver them the moment the model tries to use them? I call this guidance injection. samihonkonen.com/posts/guidan...

samihonkonen.comGuidance injection: reliable instructions for local LLMsSystem prompts don’t work reliably for smaller models. Guidance injection delivers instructions at the exact moment they’re needed instead.

Alec Solder

@alec-s.bsky.social

24 June

this is super cool! It definitely reminds me of deferred tools, but for a model without specific training for it. It also sounds like it can be implemented inference side, to prevent needing a round trip as well. I really like the ack as well

Sami Honkonen

@samihonkonen.com

The extra round trip is annoying, but significantly less so than having to tell the model afterwards to read the instructions and fix the mess it made :D

24/6/26, 10:02 am0 reposts1 like

No replies yet.