Post

Hi! SkyFeed is struggling and I need to find a solution. If you don't know SkyFeed, it's a service I built 2 years ago, enabling anyone to build and publish feeds using a visual block-based editor. SkyFeed is still hosting the most feeds on Bluesky, but that comes at a cost [1/X]

What's the general breakdown of the hosting costs? Bandwidth, Compute, or Storage?

The most expensive part is compute, I have a lot of dedicated Hetzner servers for running the queries. Storage is also pretty expensive, but that's part of the dedicated servers (nvmes), so not really an extra cost. Bandwidth is 2 TB per month max, I'm only serving compressed JSON (a lot of it)

The easiest way to get that cost down would be to explore cheaper dedicated server providers advertising on webhostingtalk, but that would realistically be at most a 30% savings. Assuming you’re at 80%+ utilization.

Donations mostly covered costs at the end of 2024, but expenses skyrocketed as I scaled everything to keep up with Bluesky growing quickly. While fighting the fires popping up in the infrastructure, I couldn't really focus on releasing new features, fix bugs or implement long-promised improvements

It felt wrong to ask for money before delivering the level of service I expect. But being a solo developer and SkyFeed not my main job made that near-impossible. I also realized that the current architecture struggled to keep up, let alone enable new resource-intensive features I promised long ago.

The first component I started to rewrite was the indexer, but I quickly realized the storage and query layers also needed a ground-up rebuild. This took far longer than expected and was the main reason for not improving the SkyFeed app or adding support for new features like video

I do DevOps, are there any optimizations we can make to lower the server costs?

The most effective optimization is really the new architecture, because the old one was built in the early days of Bluesky and never designed for massive scale. So while tuning individual feeds and optimizing some knobs did help and reduce resource usage, they were mostly band-aids.