Post

David Lindsey Pittman
@dphrygian.com

I'd only considered the ethical/copyright side of LLM training on public data and hadn't considered the server costs of scraping. Now I'm angry all over again.

8 reposts159 likes

That Guy Mike
@thatguymike.bsky.social

Its kinda hard to consider every angle of the damage caused by generative AI either through scraping or the generation itself. Theres just so many, nobody can be faulted for forgetting or not even knowing certain aspects of it all.

Banda Bassotti
@orpach.neocities.org

I'd be way less concerned if they were actually good at scraping websites. Hitting wiki sites is especially egregious, since a ton of them have archives you can grab all at once rather than hitting rendered pages every few seconds in perpetuity.

ahdok
@socksandpuppets.com

These guys are all "it's "public data" so it's fair game, but publishing something on the internet is not consent for commercial use! The default is "it's copyrighted." My site's been scraped thousands of times, and there's no way they're respecting the actual license I publish my drawings under.

ahdok
@socksandpuppets.com

Around the start of 2025, scrapers for LLM bots would be hitting my site so hard the (small) webhost would shut down (so not only my site, but about 100 other sites would go down.) - it'd reset on the hour if it was down, and the typical uptime was less than 30s before it shut down again.

ahdok
@socksandpuppets.com

Now the "solution" to this is to pay cloudflare for a service that directs bots away from the site, and that service costs more than twice as much as my hosting does - but without it, I can't have a website online with a small host. It'd force everyone onto the big providers' infrastructure.