Post

Looking for people to test the latest version of simd-sketch. It's now 2x as fast at sketching, and supports skipping over kmers containing N and other ambiguous bases (which is only ~35% slower). 'cargo install simd-sketch' is right there under your fingertips ;) github.com/RagnarGrootK...

With default params (bucket sketch with 10k buckets), it can sketch 900MB/s of input data per thread, and then can do 1 million pairwise distance comparisons per second per thread. So a full triangle of 1000 2Mbp bacterial genomes is 2.3s + 0.5s.

or with 6 threads, only 0.7s. That's with bucket size b=8. Sketching is >50x faster than BinDash[-rs], which takes at least 110s. For b=8 as above, computing distances is ~2x faster than BinDash, but for b=1, SimdSketch is again 10x faster. curiouscoding.nl/posts/simd-s...

Also note: for comparison speed, 1-bit hashes beat everything else (top-right red line here). Because each bit is an independent sample, the entropy is very high. (Whereas comparing 8-bit sketches loses some information, because if two 8-bit values differ in the first bit, the other 7 are unused.)