Post

What is the deal with everyone's bots. This is a brand new project (a few weeks ago) and they've literally crawled the whole thing and cost me > $100 in compute/memory in a few weeks. I want to allow searches but this feels like training data...

Peter Sugihara

@campsh.com

not excusing it but curious... why is it so expensive to serve requests?

John Nunemaker

@johnnunemaker.com

It’s a pretty basic site but pulls a lot of text out. On railway and they are usually quite cheap. But no requests are cached right now and unsure if it would have helped as all of them are just scanning new pages. And there are a lot of pages.

26/9/26, 4:43 pm0 reposts1 like

No replies yet.