Post

Looking for people to test the latest version of simd-sketch. It's now 2x as fast at sketching, and supports skipping over kmers containing N and other ambiguous bases (which is only ~35% slower). 'cargo install simd-sketch' is right there under your fingertips ;) github.com/RagnarGrootK...

Compared to `mash triangle` for 208 bgzipp'd cattle genomes on 4 threads, it only took 65 min vs 230 min. Used slightly more RAM (8.2 GB vs 3.3 GB), but easily worth the speedup! Mash defaults are also k=21 and 1k sketches, so not a fair comparison to k=31 and 10k (not sure about sketches v buckets)

Simd-sketch perf should be independent of k. 10k 8-bit buckets is mostly equivalent to a 10k bottom-sketch, so you're getting some more accuracy here. But since you only have 208 genomes, the 200^2 comparisons will take less than a second anyway.

I wonder how much time is spent on decompresion here; that might well be the bottleneck. I get nearly 1GB/s sketching locally, so that's 3s/genome or 200*3s=600s=10min total on a single thread, or only 2.5min on 4 threads.

Yeah; using a `.fa` human genome gives me roughly 2s reading the input and 2s sketching it. Using `.fa.gz` instead adds another 5s on top of that :/