Post

Man - AI has completely eaten up the clouds capacity. We straight up can’t do a Postgres major version upgrade because GCP doesn’t have the capacity available for a measly 64cpu/256gb mem in their “Enterprise Plus” node pool. Scheduled 2 maintenance windows and failed twice now. Ridiculous!

24/8/26, 6:34 pm0 reposts1 like

Travis Reyburn

@treyburn.dev

25 Aug

Wow - apparently Google left us in a broken state where our read replicas are no longer able to replicate. Apparently the only solution there is to blow away our read replicas and re-create them. Except now that we’ve done that - GCP still doesn’t have any resources.

Travis Reyburn

@treyburn.dev

25 Aug

So now we can’t recreate our read replicas. Which don’t even need much resources in the first place! 32cpu and 128mem in 2 regions is all we’re asking for. We even have those resources back! But now they’ve already been allocated because everything is so oversubscribed.

Travis Reyburn

@treyburn.dev

25 Aug

The future is so dumb. We can’t even operate normal software anymore because chatbot throughput is more important.

Travis Reyburn

@treyburn.dev

25 Aug

Ok now THIS is crazy. I’m hearing from our GCP rep that the reason we can’t provision any new database instances is because gcp-west1 has completely exhausted its SSD capacity

Travis Campbell

@hcoyote.bsky.social

28 Aug

"The stockouts will continue until morale improves."