Post

what if part of sync state was a small per-repo LRU cache of block CIDs? since most updates (at least with tid rkeys) stay on a few hot edges of the mst, maybe you could avoid sending most CIDs with most commits, just referencing a cache index instead

we debated a similar optimization at some point, I think why came up with it based on filesystem implementation tricks: have a small array of records be referenced directly (by path and CID) from the commit object. they would still go out on firehose, but wouldn't churn the MST structure

the firehose event would be just the new (signed) commit object, and the (new?) records. no MST nodes. the motivation was this was a high rate of small interaction records such as "likes".

I personally didn't like this because it adds implementation complexity in a bunch of places (PDS, consumers, anybody requesting a proof chain). you need to be very careful doing existence lookups, handling invalid corner-cases, etc. I just think correctness+reliably would suffer in practice

fair. where i feel like there’s value in this (and STAR) is that they are ~just alternative representation of the exact CAR contents with cheap conversion back to the standard. they could exist alongside instead of replacing, and use something like content negotiation on connect.