Post

don't forget the Elon Bros think these tin cans are about to be endowed with souls

Not just collect but interact with it, seek clarification in the light of what they learned, use that information to find the next piece of relevant info, etc. They have no model of the world that includes their own position in it.

If scraping is going to be allowed without regards to IP laws then we will just increasingly see locked down data made available only to subscribers and licensees. Everything will become silo’d in the future.

In biotech this drives me mad. You need RAs. You need bench scientists. You need data.

I would say almost all people are ignoring (or completely unaware of) that fact. So much hype around how AI is going to make new discoveries and no, it is not. It is a club DJ mixing existing songs together.

My father asked me why AI couldn’t just take over monitoring the weather and I was going “Okay. Let’s start with how exactly AI is going to place the weather stations…”

The people who use LLMs for travel planning are already running into this in that those LLMs can only exploit info a human has ready put into an accessible online system, and travel information is inherently constantly volatile and shifting

The self-pwning delusion of people like Musk and Kalnick is that somewhere in that mountain of data is a connection someone has missed, but LLMs can, with their massive connectivity of words alone, ferret out the one detail that grants FTL, or immortality, or a true moral basis for selfishness.

As part of this discussion, I was thought-experimenting hooking an LLM up to a robot. I had this weird vision of the system creating a zork-like interface.

Discussed this recently. My buddy codes & told me AI will take over EVERYTHING. Because it codes really well. I said that it codes well because computer guys write what they know into AI & that it’s shit on anything that’s not already written. I could see that he’d legit never considered this.

The biggest limitation of LLMs is that the data ingestion rate they require to maintain relevance is greater than the rate humans produce data. They gobbled up all the information on the internet and need another internet's worth of new data NOW

It's extremely dependent on that in the real world, but also just in the computer world, we still have to send in crash reports and explain what happened when an app crashes, and then the app companies don't have enough people to even solve these problems.

I mean, you’re 50/50 on this, I think. Anything that does exist in online readable form, an LLM can consume, and their existence relies on us having done that to all relevant info consistently for 2 decades. However, they’re killing the incentive systems that made us do that so the future is unclear

Well, Grok uses Twitter for that, it works great! 🤪🤮😱😬

Doing this is basically creating Asimov level androids

Seems so obvious nobody thinks about this. No wonder the LLMs always seem to be "looking" backwards. No wonder they seem to be so *conservative*... And we're surprised by this? And take it to mean "conservatism" *must* be "right" because "the all-knowing machine" says so?

I used to work for a company that produced some creative work for clients. They'd use "design sprints" to have a group of people work together to hash out product specs. Invariably, the group effort tended toward consensus--meaning the most obvious and mundane solutions.

I mean good luck with that I guess but we’re still not very good at mobile robots

I should admit that I am a weirdo that has an unabridged dictionary to look up words. I like the glancing at pages as I'm heading out the word and seeing other words that interest me. I can't get that by looking a word up online.

and an assistant can't know what he will find interesting and/or useful. And that's with a person so I'm not sure how AI can do that.

There’s also not enough information in the universe to make these models not suck shit.

I like the framing. It suddenly makes sense of the "we see you bought a politician's lies, here's a dozen more you might like"...

it's like they never even played mgs4. only half joking.

From a development standpoint, the toilet thing should be easy to fix. They have a huge amount of data to show that most people buy, at most, 1 toilet and thus it is very unlikely that you would buy a second one It’s actively strange if this is a real and ongoing issue with one-time purchases

Also, "Look I know you live in Greenland, but there's a fast food place on Main Street, USA that you just have to try!"

We have *actual systems* for that. Why are 99% of Gen AI use cases looking like trying to hammer a nail with a banana?

Not to mention the feedback loop of AI-generated content being fed back into AIs until you get an oroborous effect going

Also “developing the models” is not going to make LLMs be something other than LLMs … which are, by nature, only simulators of what factual information *might* look like

They’re free riding on the entirety of human cultural production and it’s not ever going to get any cheaper than it is now to get access to new information.

Incestuous AI training 🤢

It’s also a major problem that they’re now generating data created by absorbing OTHER data generated BY themselves & other learning models. Dogs lapping up & regurgitating their own vomit.

So we come to the "xerox of a xerox of a xerox" problem, or the "google translate back and forth and back and forth" problem.

SCP weirdos will poison wikipedia for free forever, best training data in the world!

The models currently have 3 independent axes on which quality grows logarithmically: 1) pre-training data 2) model size 3) test-time compute Only point 1 is dependent on new data, and they haven't really tapped into video data much.