Amazon Science on Mythryl: “When LLM judges agree, the right question is why. Shared prompts…”

When LLM judges agree, the right question is why. Shared prompts, model families, or training lineage can make a majority look stronger than it is. Dependence-aware aggregation via Ising models accounts for this, improving accuracy 9–14% over weighted majority vote.

amazon.scienceWhen LLM judges agree, should we believe them?Discounting the opinions of LLM judges with highly correlated outputs ensures that panels of judges reflect a true diversity of perspectives.

28/8/26, 6:29 pm0 reposts1 like

Post