Post

Hate to say it, but I might need to stop my indexer for docs.surf. It’s. Fun little project, but the costs are slowly creeping upwards as standard.site continues to grow. Tap listening to every single record just eats up way too much egress.

with tap bandwidth do you mean ingress or egress? the full firehose does add up to a lot of ingress every month, but that is no-cost with many hosting providers. in theory you could split out just the tap part to one of those, and keep the rest of your setup where it is

Have you looked at Durable Objects on Cloudflare for listening? It shouldn't cost too much to listen to the Jetstream from there. Ingress and egress are free. You just pay for the DO minutes, which would be within the free tier.

For a while I moved the tap instance to my home server which did work, but eventually the ingress was so high my home internet was getting throttled (kids and wife were not happy lol).

Looking at the rate of new posts on docs.surf, that would be pretty minimal egress? Or have I misunderstood?

I did some deeper digging around my billing for Railway which has been the primary cost, and its kinda split between egress and CPU. Currently switching to a durable object on cloudflare that listens to jetstream which should help a bunch.