Post

In a horrifying abuse of modern computers, here's a shell one-liner you can run on macOS to calculate the number of days between two dates in the most complex way imaginable - using uv and Deno and the Pydantic AI Python sandbox MCP server simonwillison.net/2025/Apr/18/...

7 reposts86 likes

This could be an interesting, very abstract way to evaluate LLMs other than with usual evaluation tools. Let the LLMs generate code for a problem ("how many days..."), execute it and check their output against the output of known working code/libraries/etc. Or is this already standard?