Post

Access will be granted to you when the gods’ blessing arrives

0 reposts14 likes

FYI the ARC-AGI-3 result is highly sus, given that Opus 5 was around 20-30 but hit 100% with 2 different harnesses. i basically discredit the entire benchmark at this point

it's 99.9% because: - scoring isn't actually linear - it's using a different harness. arc-agi-3's default harness is kinda bad and unrealistic, while the other one is a normal one. still though, with the default harness, it still gets about 60%.

Yea, they’re doing their blog post thing of testing on ARC-AGI-3 with a different endpoint and retaining reasoning and using their internal compaction. I think that’s fair but not when comparing to all the other models still being nerfed with those issues