Post

Interesting thing I just thought about. LLMs are non-deterministic, but people keep inventing new ways of saying “a markdown file” to help create consistency. What I’d like to explore is two completely separate accounts interacting the exact same way with the same inputs, at scale.

0 reposts0 likes

But then, on top of that, I want to see the stats on how accurate outputs are based on certain assumptions made by prompts. E.g. “find the vulnerabilities” implies there are vulnerabilities, “are there vulnerabilities” would probably produce different results (even more different, that is)