jayk56.bsky.social

jayk56.bsky.social

@jayk56.bsky.social

4 followers8 following43 posts

the answer was not quite, and >90%...times they are a changing

jayk56.bsky.social@jayk56.bsky.social

what does it mean when ALL sota models score 95%+ with different cli's? (codex, claude code, etc) aider seems to be a drag on their performance now, but we're back to the bigger question of can these be 'plugged in' to software workflows and what percentage of code will they be writing by EOY

jayk56.bsky.social reposted

what does it mean when ALL sota models score 95%+ with different cli's? (codex, claude code, etc) aider seems to be a drag on their performance now, but we're back to the bigger question of can these be 'plugged in' to software workflows and what percentage of code will they be writing by EOY

jayk56.bsky.social@jayk56.bsky.social

what does it mean when a model can score 95%+ on aider's polyglot benchmark? how does it change what you can do with it vs the 70% gemini pro is at now?

as a proud American I'm disgusted by the cowardice the republican party in congress has shown, it's up to us to remind them who they represent. republicans have shown they are worse than those they hate, they have: NO budget NO beliefs NO backbone saturday we remind them we all have #NOKINGS

jayk56.bsky.social@jayk56.bsky.social

why I'm protesting: we have a government official trying to claim king like powers to punish the out groups and playing chicken with our allies. I reject this. team trump: RESPECT our people RESPECT our constitution RESPECT our allies #handsoff #handsoff2025

currently using this to vibe code an mcp server. i'm getting a crash course in javascript tdd and packaging. :) also going to use pre-commit hooks more in my other projects, i knew it was possible but the reward didn't seem to match the effort of implementing. llms lower that bar, just like for tdd

also worth noting, i don't expect this server to be used for anything, but needed 'something' to throw the 'agent' against. code is likely to be trashed after this evening, but it's slop with a purpose

what does it mean when ALL sota models score 95%+ with different cli's? (codex, claude code, etc) aider seems to be a drag on their performance now, but we're back to the bigger question of can these be 'plugged in' to software workflows and what percentage of code will they be writing by EOY

jayk56.bsky.social@jayk56.bsky.social

what does it mean when a model can score 95%+ on aider's polyglot benchmark? how does it change what you can do with it vs the 70% gemini pro is at now?

please help fill in some blanks! openai: really gets the ergonomics of an ai chat interface (ChatGPT) anthropic: really gets the ergonomics of ai agents (Claude Code, MCP) google: really gets the ergonomics of __________ deepseek: really gets the ergonomics of ___________ #ai #llms

using claude with artifacts in the web client, I'm not surprised claude code gets this nickname

Simon Willison@simonwillison.net

I'm now calling Claude Code "honey badger" on account of its voracious appetite for crunching through code and tokens looking for the right thing to fix simonwillison.net/2025/May/23/...

Another fun fact is the new Gemini 2.5 Flash (non-thinking) preview model performs on par with Claude 3.7 Sonnet (non-thinking) on this benchmark for 1/10th the price...

jayk56.bsky.social@jayk56.bsky.social

Gemini 2.5 Pro is a beast for the price and, given 8 tries per problem, scores 93% on the Aider benchmark. o4-mini (high) was not far behind, but cost about 30% more to score 91%. Sonnet 3.7 without thinking is able to average 87% but cost about $45 per run (more than double the cost of gemini)

Gemini 2.5 Pro is a beast for the price and, given 8 tries per problem, scores 93% on the Aider benchmark. o4-mini (high) was not far behind, but cost about 30% more to score 91%. Sonnet 3.7 without thinking is able to average 87% but cost about $45 per run (more than double the cost of gemini)

thankfully, the people and USA continue to show it is strong enough to rebuff the would be dictators and totalitarians. building your communities only strengthens that defense, and it helps remind us that we're all in this together.

Things I didn't have to think about 6 months ago: - Do I need to pull up expected purchases to avoid paying 30% more? - Will I be harassed coming back from a vacation out of the country? - Will congressional republicans grow a spine before it's too late? what are some new worries you didn't have?

why I'm protesting: we have a government official trying to claim king like powers to punish the out groups and playing chicken with our allies. I reject this. team trump: RESPECT our people RESPECT our constitution RESPECT our allies #handsoff #handsoff2025

Hundreds of SacTown locals showing up today to protect veterans, retirees, and science and protest oligarchs carving up the American public's wealth. let's turn those honks into action with the Hands OFF! protest April 5th. #SacTeslaTakedown #TeslaTakedown #StandUpForScience #HandsOff

wondering today-if LLM are the computers-when are we going to see the 'office suite' of AI get launched, is this going to be limited to Gemini/Apple Intelligence integration? Is it what OpenAI is building out to serve the existing chat customers? operator, search, deep search, canvas/projects, etc.

VSCode + AI = ultimate text calculator! I'm sure this has been a feature for a few releases, but I wanted to merge a file from my clipboard into an existing file, BUT only change a couple classes. Using the 'Compare File with Clipboard' I was able to select the individual sections to apply. ezpz

I might be struggling with prompting, but o3-mini-high provided documents 1-3 and o1 provided document 4. Here is a summary of what I was noticing (as stated by o3-mini-high grading the responses)

Feeding that back into o3-mini-high and having it regenerate seems to have gotten it just about on par with the o1 response. You have to be a bit more detailed in what you want from o3-mini than o1, but with the higher rate limit you can generate a few times

How these new reasoning models are prompted is still so important that domain expertise still gives you a massive boost over the average person in getting a 'good' answer from the models. The next phase is probably looking at how to get better results with less specifics in the prompts.

OpenAI's Realtime API feels heads and shoulders above Gemini 2.0 Flash live, but it's also possible it's just easier to get working examples going. I was comparing an example react component with 4o-mini against Google's example python cookbook with gemini 2.0 flash and flash just felt shallow..

You are up to date.