This is actually a massive result that people are really all sleeping on. We can just start over from zero and, with the right language design, build a programming language that the models can write perfectly without a training corpus.
Expanded the Vera benchmark to 6 models, 3 providers. Kimi K2.5 got every Vera problem right while only managing 86% in Python and 91% in TypeScript. Flagship average Vera and Python level at 93%. Yet there is no Vera in their training data, no examples on GitHub or Stack Overflow. veralang.dev #AI
0 reposts2 likes