Looking for people to test the latest version of simd-sketch. It's now 2x as fast at sketching, and supports skipping over kmers containing N and other ambiguous bases (which is only ~35% slower). 'cargo install simd-sketch' is right there under your fingertips ;) github.com/RagnarGrootK...
Post
github.comGitHub - RagnarGrootKoerkamp/simd-sketch: Compute bottom-s sketches and s-buckets sketches, using simd-minimizers crate.Compute bottom-s sketches and s-buckets sketches, using simd-minimizers crate. - RagnarGrootKoerkamp/simd-sketch
Compared to `mash triangle` for 208 bgzipp'd cattle genomes on 4 threads, it only took 65 min vs 230 min. Used slightly more RAM (8.2 GB vs 3.3 GB), but easily worth the speedup! Mash defaults are also k=21 and 1k sketches, so not a fair comparison to k=31 and 10k (not sure about sketches v buckets)
I wonder how much time is spent on decompresion here; that might well be the bottleneck. I get nearly 1GB/s sketching locally, so that's 3s/genome or 200*3s=600s=10min total on a single thread, or only 2.5min on 4 threads.
0 reposts0 likes