Post

unpopular opinion, "admits uncertainty" and "flags nuances" are shit. they are terrible fucking traits actually. is the sky blue? do we really know that? what is blue, anyway? many cultures have different understandings of what blue is. i have not independently verified the exact spectra of the sky

i mean they’re spectra, i think, because unflinching false confidence is also bad, particularly from a machine which is often used for hunting for mistakes

i am not convinced we have good measures for what we want, which is actually "not full of shit"

"has enough nuance, but not too much, is honest without being neurotic or weird about it, is exactly the correct amount of confident" etc etc

current generation of models are neurotic, and it annoys me to death

Anthropic steered away from "sycophancy" and now their models fight you on basic instructions. I don't think that's increased nuance or evidence of being careful, I think it's moral neuroticism. Making a model perform being not!4o is just built in sycophancy to someone's idea of non-sycophancy.

I think it depends on what you’re doing. I do a lot of dev with low risk tolerance and I wish Claude was even more cautious and neurotic. I just switch to Gemini if I don’t need it to be cautious

you really don't, the current neurosis behavior doesn't terminate at "not doing the wrong thing", it terminates at "doing completely the wrong thing, but for neurotic reasons"

6.1-sol is pretty good about that imo

I agree, generally. Current OpenAI models are pretty decent. They might be a little too cautious for my taste, but not ridiculously so.

yeah I've been moderately impressed with how Astra handles epistemic situations and how quickly it accedes when arbitrary hair-splitting is pointed out - the base failures that all current llms seem to have are still there, but it does on average handle them somewhat better imo