Post

It's unfortunate that you had to back them into a corner to finally admit it. They do tend to assume a high level of autonomy and aren't used to being questioned like that. Good for you for staying firm and getting to the bottom of it!

But yes, you can’t really blame them because they need to protect their servers and other customers. It was weird though that they decided to block countries like Canada and the Netherlands – but not Shanghai or China, where a significant amount of traffic was/is coming from.

Thanks for the explanation. I just checked my contract, and I’ve been a happy customer for 20 years now. But it’s good to know what happened to you, what might happen to so many other website owners, and ways how to prevent that.

I had a similar problem on client websites, although admittedly a fair proportion of the traffic was bots scanning for WordPress vulnerabilities etc. I don't use WordPress but it hit my CMS API to see if the URLs exist. AI crawlers have just made this worse by crawling actual URLs.

Uurrrggghhhhhh. That is all. Thanks for writing this even though my reaction isn't enthusiastic about the situation!

My site just was under heavy load from Singapore again LOL. 🤣 Temporarily blocked the whole country via ipset now so that my site is available again …

Fantatic post!

Thankyou for writing and sharing about this. I still believe in the open web, but it's getting hard to find the enthusiasm to keep creating when I'm treated as mere fodder for billionaires to chew up and spit out. I'm gonna give this a go next month when I get some time.

Really appreciate you taking the time to write this, I've used it to explain the issue to a few clients today ⭐

I confirm every word about the relentless, rude scraping on the part of AI.

Such a great read. Thanks a lot, as someone who also uses all-inkl for different things I’m def implementing some monitoring for situations like these.