Post

Something we've found helpful when testing prompts for MetaMonster: have the LLM output it's "rationale."

While LLMs aren't really thinking, this can help you understand how its model may have "interpreted" your instructions and at the very least give you ideas for what to tweak to get the output closer to what you're looking for.

8/6/25, 4:00 pm0 reposts0 likes

Andrew Askins

@andrewaskins.com

8 June 2025

Maybe it's weighting a certain word heavily, or associating it with something you didn't consider. (at least I think that's what's happening) The new UI makes it easy to generate this at the same time as the output from your prompt inline, and then review them at scale.

Andrew Askins

@andrewaskins.com

8 June 2025

That's another key to prompt engineering: LLMs are probabilistic, so you need to test prompts across a decent-sized dataset to get a real understanding of how well they work.