Post

Why not? I often run tiny ~2GB models on my laptop which somehow contain quite a lot of factual data - though it's a roll of the dice if you'll get a correct answer to anything out of them

Swiss cheese has been my similar analogy of late: it tastes like cheese and once melted (compressed) you never really know where the holes were. Like @antirez.bsky.social, I think that this kind of compression should be lauded as a way to freeze and revisit ideas in our collective history.

I manually add alt test to all of my images - for things like this it's easy, I just copy and paste from the underlying blog entry - for others I often use Claude for the first draft simonwillison.net/2025/Mar/2/a...

LLMs are like an extrapolation engine based upon a lossy encyclopaedia. I like that analogy.

LLMs are like an extrapolation engine based upon a lossy encyclopaedia. I like that analogy.

word calculator still my favorite, especially as context has grown longer

LLMs don't snapshot anything. They are probability weights used by gambling machines that are really good at mimicking their training data, but will lie to you just as easily as they'll tell you the truth because they don't "know" the difference.

That’s a good one. I have been calling them “spooky fuzzy databases”

Wow! Gotta noodle on this. But hot take: You could write a fuzzy SQL query and w/iteration get a decent result...but it's very unlikely to get spooky. You could write a fuzzy/bad/incorrect LLM query and get a decent response to the query you ~should~ have written, w/little to no iteration. Spooky!

Would that make cloud models "spooky fuzzy databases at a distance"?

I like this framing, and they also do a lot more than information retrieval, they synthesize information in the moment with a complexity we don't see in prior knowledge artifacts -- spatial metaphor is one way I try to think about how the process is more than just lookup bsky.app/profile/ryan...

I like this a lot. There are prompts that are the equivalent of line art for JPEG.

An encyclopedia tries to only contain true statements, so even that is too strong a claim.

Trying to explain to many people who have gotten into the hype that there are both things it is good for and things that it is not good for is the trouble. Back in the day we were trying to derive fact without language using Prolog, now we have language without a concept of fact.

Yes it is, but they probably are not using the same data they used for training. Most likely, publicly crawlable content in search, and for training, they use torrented LibGen... If the search tool could use LibGen too, it could potentially be even more accurate, as it same data used for training.

Everyone's happy to bend the rules while the boom is on and money is flowing. They're riding the train. It'll change if the money dries up. If you're not an advocate of IP rights you are free to public domain. Having your work lifted and used without consent or compensation is not right.

Through your advice on how to use them: give it facts to reason about. Is generally good. Don’t over index on theory. We are at the phenomenon collection phase

I've thought, drunken sage. Or just a a highly condensed memetic package.

It's a vastly indexed but imperfectly directed collectania/dumpster of human knowledge.