Post

⚡ I finally dropped a new batch of optimizations for the fast Python IPLD library. Now we can read atproto repositories and firehose commits 30% faster! My local benchmarks show 200 MB/sec. GitHub Release: github.com/MarshalX/pyt...

8/9/24, 12:26 am4 reposts19 likes

Ilya (Marshal)

@emily.space fyi

Amazing work, thank you! 😊

Ilya (Marshal)

@marshal.dev

8 Sept 2024

@retr0.id`s kindly crafted benchmark of Python dag-cbor parsers has been updated to include new libipld update! github.com/DavidBuchana...

github.comGitHub - DavidBuchanan314/dag-cbor-benchmark: Test data for benchmarking dag-cbor decoders and encodersTest data for benchmarking dag-cbor decoders and encoders - DavidBuchanan314/dag-cbor-benchmark

Kuba Suder 🇵🇱🇺🇦

@mackuba.eu

8 Sept 2024

Oh, interesting, I'll check how the Ruby one I use compares 🤔

David Buchanan

@retr0.id

8 Sept 2024

idk the best way to integrate it, but if you somehow PR ruby support I'd accept it (maybe I should have each benchmark run as a standalone script that returns timing info on stdout)

Ok will do 🫡

pls ping me with results

Ilya (Marshal)

@marshal.dev

8 Sept 2024

🏎️ Performance comparison. Before (v1.2.3) VS After (v2.0.0):

Nice!! 👏

I'm guessing it's not a coincidence that @snarfed.org just switched to this one ;)

Ilya (Marshal)

@marshal.dev

8 Sept 2024

i swear this not related XD. i was blocked by strange bug in rust toolchain. so i decided to postpone release for a while. nothing fixed for last ~5 months. so i decided to use older Rust version. the reason why i do it now because of higher bsky network activity that we need to process :)