Post

Larry Osterman

@larryosterman.github.io

New phrase I just came up with in the context of AI evaluations: "Without the miasma of correctness..."

17/9/26, 11:41 pm0 reposts9 likes

Larry Osterman

@larryosterman.github.io

17 Sept

The context was thst if I asked copilot to wektr an evaluation, the evaluation always included a particular best practice, but if I removed that best practice, the evaluation model never implemented the best practice.

Larry Osterman

@larryosterman.github.io

17 Sept

My hypothesis is that copilot absorbed the best practice from the contents of the repo, but the evaluation (which runs in an isolated environment) didn't have access to the "miasma of correctness". And I just love that phrase....