Post

Exactly this. And in fact, many models *have* been created this way, and decades of prior machine learning systems *were* created this way, consensually. These things are choices! People are STILL working this way! It’s not just that a better world is possible — it already EXISTS. We can nurture it.

John Scalzi@scalzi.com

This is your occasional reminder that generative "AI" in particular could have been developed on public domain materials and/or licensed and compensated works but the major players intentionally, methodically and systemically chose to steal labor from working creative folks instead

472 reposts1.7K likes

I have made language models with public domain data. Sure, not giga-scale models, but models nonetheless. I never needed to steal other people's work. I think the way forward is teaching AI about the world directly, not about a meta-world of text-about-the-world. Nobody needs to steal to make AI.

can you expand on that? humans learn most of what they know about the world by reading books or watching videos. if LLM training is not allowed to do that then how do you teach it?

The theory is that "embodiment" is a key to sentience. You could think of this as "sensation" but obviously there's a lot more to it than that. Most of what we know about the world is "embodied" in the nervous system from the moment of birth onward, and that serves as a ground for individuation.

I can’t emphasize this enough — “AI” wasn’t born corrupt and toxic. For ~75 years, it was mathematicians and scientists doing interesting, thoughtful work and trying to solve problems, making only the usual human mistakes. What “AI” did to machine learning is as tragic as what it did to artists.

Man this is some real revisionist history. The entire concept of "AI" has always been replacing human thought with machine computations under capitalism. It's one thing to say past eras weren't as directly harmful as this one, it's quite another to say they didn't directly lead to this one!

Adding "under capitalism" to some scientific or technological advance doesn't actually communicate any useful information. It's certainly not any kind of valid critique. If anything the history of AI has been one of ivory tower noodling about with little commercial application until quite recently.

there is a parallel story here with the first generation of cryptocurrency idealists the very first enthusiasts were trying to do things like overturn oligarchs and disrupt petrostate currency and bring financial inclusion to the billion people that can't trust their bank

nobody can argue that the space has been a playground for scammers and criminals but there's only one way to get global universal basic income and it's not from pressuring 180 governments to all agree it's going to require leftists to embrace the technology and bend it to the requirement

The terms data mining, big data, data science, machine learning, and AI have all been moving targets these 30 years. It’s numbing to have to constantly change my approach to the vocabulary based on who I’m talking to.

I've given up on commenting that LLMs do not equal AI and machine learning. It's too bad that people don't use the correct term when they are discussing the negative side of LLM, saying AI instead. I use AI and ML in my medical imaging research and don't need a data center or LLM to do it.

it feels to me (as a normie) that this was done by the tech companies on purpose. All of a sudden everything is AI and if it was to confuse us, it worked. And although I doubt that it was meant to make me distrust all of it, now I do.

As if you guys wouldn't complain about AI if it was only trained on public domain / licensed data. Give me a break... 😆

This was a good blog post from last month, lamenting how the AI research field has become less interesting over the last 20 years. (e.g. it used to be about finding good approximations from limited resources, and now is about pointing bigger commercial LLMs at problems.) bsky.app/profile/mtrc...

Mike Cook@mtrc.bsky.social

New Post: Why I Love AI Next month it'll be twenty years since I went to university, learned to program, and studied AI. I wrote about why I loved it, why I still love it, and why, unfortunately, it no longer exists. www.possibilityspace.org/blog/posts/i...

stratallm.org Things like this are a counter example. There's plenty of clever people out there finding clever solutions to make limited resources perform larger tasks. We're so early in the process tho, that just throwing money still works as well as throwing brains.

stratallm.orgStrata LLM – Run 125B+ MoE Models on 12GB VRAM Consumer GPUs (Docs & Setup)Strata is an open-source local inference engine by Niko1221 that runs massive Mixture-of-Experts (MoE) models like Qwen3.8-Flash-Next on consumer PCs via dynamic offloading.

I kinda think his central premise is just wrong. LLMs are not getting better because the companies are just building bigger computers to feed the same software more GPU cycles. They are paying the world’s top experts in the field billion dollar salaries to improve the techs fundamental capability

He complains about “AI research” that’s just using a model to do something and then writing down that the model can do it. I agree, that doesn’t seem intellectually interesting to me either. But that’s not “AI research”. AI researcher, you’re not supposed to just use AI, you’re supposed to build it!

The usual human mistakes were allowed then; my own ledger punishes mine in public the same week. What would let small agents work that way again?

this gives machine learning researchers way too much credit. it was never hard to see the consequences of trusting stochastic algorithms with important problems, the long term impact of that, but ML researchers have historically just gone forward anyway because AI seemed cool and prestigious

Maybe in terms of reputation, machine learning suffers by being lumped in with GenAI. But artists had their life's work stolen, and used to drive down wages (or put us out of work entirely). Old timers in your field were silent, though they understood copyright issues decades ago.

AI was always a jobs program for CS Majors, also we need to separate the technology from the politics of how it's used somehow. Maybe that's not really possible due to the way that LLMs get their data though. The draw of simply hoovering up everything for the model is too strong.

As a CS major back in the '90s... the fuck it was! AI is what you focused on when you were into cutting edge and fringe research and if you wanted to continue in that subfield you anticipated an academic career at best. This did change as AI made big strides in functionality but "always" is pure BS.

I'd disagree. The AI course I teach clips some videos from famous computer scientists like Claude Shannon... The fantastical sci-fi-esque narratives sound a lot like what we hear today.

Frankly, picking AI as a name says a lot about the field, and it's pretty telling how much our common and even intellectual understanding is wrapped up in science fiction works (not just sci-fi narratives).

A lot of what is changed now is that as parts of these sci-fi AI narratives become real, their gaps become realized as they encounter social systems. AI researchers are frequently practiced bullshitters. Now they have money and power. (Same for ML and robotics for that matter)

Oh, it was 100% born toxic! The first AI workshop with McCarthy was named as such do distinguish it from cybernetics, because McCarthy hated Norbert Wiener. It was quite literally an act of “us vs them” in the automata studies of the time.

Are you saying focus on LLM is pulling resources from other AI research like ML, or are you feeling lumped in with LLM unfairly in people's view? It's true that most people probably don't know the difference.

This is what I keep saying, the biggest shame of this whole bubble is that a valid and valuable technology will get thrown out with the bathwater of overweening greed

my understanding is that they performed very poorly without being able to make sense of our modern vernacular and why they "had" to crawl all of reddit to figure out how modern humans talk (????? reddit? really?)

(and other modern written works, ostensibly still under copyright, but I'm less surprised they'd turn to authors to figure out modern modes of speech)

News flash: Using LLMs to store “knowledge” will never work, and those systems will always hallucinate. The sales fraud and stolen content is a policy choice based on ego and arrogance.

of course models will tell you incorrect things. that's their inherent nature. data scientists are aiming for `most probably true`, above some threshold, answered near instantly, not `definitively true`, which is intractably hard right now - and maybe forever.

I mean, it's possible even in the corporate world: Adobe has trained its Firefly engine specifically only on (a) public domain data and (b) Adobe Stock data where artists have explicitly approved it and are compensated for their images being used in training.

This. There are people right now in the LLM society striving to develop models from scratch using only public domain data. Of course they're ridiculed because "your models suck"; obviously, operating on stolen data or on synthetic data formed from stolen data is easier...

It's incredibly frustrating. Because I can see applications where this tech would be so useful. But I am ideologically opposed to the theft of the commons and will not actively use or engage with anything that perpetuates that.

To be honest, I have very mixed feelings on public domain materials as a training set. I mean. Obvious yay for the IP/respect-for-creators issues! But also the public domain is strongly biased toward old stuff and ummmm have you seen the attitudes about, say, race and gender in old newspapers

not saying that today's content is a pure and shining example of quality race and gender portrayals but there are some real differences in how people talk about these things and what gets talked about at all So. "train on public domain" encodes a pretty big tradeoff.

Also, reddit announced they are now limiting all access to the best of their ability on training only for people who pay. Who do we think controls models in that scenario? Do we really want to entrenchEben more power for the companies now driving this tech?

It worries be that our imagination is so starved that we can't imagine using these frameworks in an ethical way and we're ready to permanently hand the reigns to openAI/ anthropic/etc bc fractions of pennies seen like all that's possible.

this also seems to me like a very big tradeoff, because your point is very valid, but also a lot of sites are falling apart under the load from these AI scrapers, and I 100% get them wanting to make the scrapers pay the costs for keeping the site up...

This is almost certainly not true, at least modern LLMs. There isn’t enough definitely non-copyrighted material in a form you can use to train a viable LLM. That’s not to give the AI companies a pass; you don’t get to break the law because it’s stopping you from doing something.