Post

There is so much to gzip. It's a perfect microcosm of the experience of Computering. Fractally complex and clever. Hamstrung by the limitations of computers at the time it was invented. Dramatic, backstabby origin story. Completely unused extension points. Both prescient and myopic in its design.

Interesting story, Jon. Would you list the main points of interest in gzip for us, the nom-specialists, reading your posts and wondering how exactly gzip looks so fantastic? I’m a big fan of UNIX software, they indeed carry a lot of wisdom through the time. I’d love to know a bit of this story!

Most of this is interesting to me because the world of containers and packages is mostly shuffling gzipped tarballs around. The culmination of this fever dream is oci.dag.dev I extracted some relevant bits to github.com/jonjohnsonjr... (if go is legible to you) Also github.com/google/crfs

I also wrote a poem about all this that only exists publicly in the subscriber-only content of an episode of the now-defunct Ship It podcast so I should probably publish that somewhere else.

Another very cool project is rapidgzip and ratarmount github.com/mxmlnkn/rapi... gunzip is normally not meaningfully parallelizable but rapidgzip attempts to speculatively perform partial decompression at likely DEFLATE block boundaries then stitch them back together once enough window is decoded.