Post

1/6 What does MSA Pairformer learns about evolutionary relationships? @yoakiyama.bsky.social et al achieved impressive contact & variant-fitness predictions w a fraction of ESM2's parameters; we wanted to better understand the mechanism arcadia-science.github.io/2025-phyloge... 🧪🧬 🖥️

2/6 Their key innovation is query-biased attention, where the model weights MSA sequences by relevance to the query. We tested whether "relevance" maps onto phylogenetic distance by correlating sequence weights with evolutionary distances across thousands of protein families.

3/6 The model consistently upweights evolutionarily close sequences and downweights distant ones, despite never being trained on this. There's also a division of labor across layers: layer 11 acts as a strong phylogenetic filter, while others contribute more modestly.

4/6 But surprisingly, encoding phylogeny ≠ leveraging it. Comparing learned weights against uniform averaging for contact prediction, the aggregate improvement was marginal. Per-MSA, uniform averaging outperformed learned weights about half the time, sometimes by a large margin.

6/6 As the field shifts toward MSA-based architectures, the quality of input alignments becomes paramount. MSA Pairformer encodes evolutionary signal well, but future architectures might benefit from learning when to trust that signal.