Post

Notes on putting the new Gemini 2.5 Pro through its paces - I'm impressed: it did great on image recognition, audio transcription and returning bounding boxes for creatures in a complex photograph, plus it rendered a pretty solid SVG of a pelican on a bicycle! simonwillison.net/2025/Mar/25/...

11 reposts132 likes

I've used OpenRouter's free 2.5 Pro hosting to generate some Swift test code, and the output was considerably better than new DeepSeek's and Claude's. Not one-shot because that would've needed more context (big project), but good enough to actually find a bug in the underlying implementation.

maybe you've already written about this somewhere, but where did you come up with the idea to test models with the task of specifically drawing a pelican riding a bicycle? I love it, by the way, it's a great through-line and calling card.

Updated my blog post to note a couple of extra things I had missed: the output token limit of 64,000 is a huge upgrade from the 8,192 of previous Geminis, and the knowledge cut-off date of January 2025 is an improvement on Gemini 2.0’s cut-off date of August 2024

The 2.0 family is affordable and has large context windows. I feel like price/performance of these models is great for at scale usage.

After trying out Gemini 2.5 Pro experimental 03-25 yesterday on a real couple of hours project it will be my go-to model rather than Claude from now on. It took me a large number of iterations, a false start but I managed to get something working in the end. For that problem nothing came close.

i've tried it on a couple things, so far really good, and 1M context, 66k output is epic. new deepseek appeared promising but seems like there's issues serving it. wonder what the pricing will eventually be on 2.5? feels like a notable step up over the already good 2.0 family.