Post

Stoop calling it “hallucinating”. “Making shit up” is a lot more accurate.

No. It's hard to find a human metaphor for LLMs because anything you pick will come with incorrect anthropocentrism, but one large benefit "hallucinate" has over "making shit up" is that the latter implies an intent & voluntary control that aren't there. Holds even stronger for "lying" btw.

this is the exact reason *to* call it making shit up; "hallucinate" implies a lack of control, and even illness. but it is code which does what it is designed to do: draw conclusions. purposely. and since it's also extrapolating, we also call that 'making shit up', especially since it's inaccurate

Yeah. I’ve hallucinated some stuff that turned out to be sensible and useful in the long term. Lying is just lying.

Right, and what LLMs do isn't lying as that implies a deliberate distortion of a truth LLMs have no notion of. "Bullshitting" is closer. Also, hate to contradict further but "sometimes is useful on accident" is a feature of hallucinations that *does* fit LLMs quite well.

People have been package-name squatting for ages. Different spellings of popular packages are a particularly ripe target.

hell, look at the leftpad incident

I always get worried in particular about packages whose common names are different from what you actually type to install them (for example, to install PyTorch, it's not "pip install PyTorch" or "pip install pytorch", but rather "pip install torch).. It's a really common situation...

A close read of this article reveals a dude saying “here’s a hypothetical threat that my company happens to sell a solution for.”

Yay. 🤮 can’t wait until AI-generated job reqs list expertise in fake software as a “Must-Have”

Literally a month after some code bases dropped, there were employers asking for two years experience with it, despite it not being publicly available until then.

Design too: “Must have 5 years experience with Figma” back when it had just hit the market

Yeah basically a lot of LLMs don't actually "know" much about coding. But they "know" a function/procedure could theoretically be written to do anything. So when the LLM gets stuck it just fills it with a call to a non-existent piece of code like GetDeusExMachinaHere() or whatever.

LLMs are probability engines. If you prompt it on a word or sequence of words that occurs a lot in the training data, they'll spit back words that tend to follow those initial words the vast majority of the time. If you do something to get it down a low-probability path or feed it a sequence that

doesn't occur (much) in its training data, it's still digging around for words that have some probability of following the words you've prompted it with. Except you've kicked it into a low probability space so by definition nothing it's going to find has a high probability of occurring in reality.

For some, using a locally installed LLM not connected to internet can be a solution. Maybe it will not be 100% up to date but does it really need to be? My biggest worry is malware infected libraries.

This makes the hallucinations worse.

Not a solution. A locally installed LLM is exactly as likely to hallucinate nonexistent package names as an internet-connected one. It's the internet connection of the *compiler* that allows the malware to enter. And all modern development environments require an internet connection to run.