%20--%3E%3Cmetadata%20xmlns%3Ardf%3D%22http%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23%22%20xmlns%3Axsi%3D%22http%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema-instance%22%20xmlns%3Adc%3D%22http%3A%2F%2Fpurl.org%2Fdc%2Felements%2F1.1%2F%22%20xmlns%3Adcterms%3D%22http%3A%2F%2Fpurl.org%2Fdc%2Fterms%2F%22%3E%3Crdf%3ARDF%3E%3Crdf%3ADescription%3E%3Cdc%3Atitle%3EShadows%3C%2Fdc%3Atitle%3E%3Cdc%3Acreator%3EDiceBear%3C%2Fdc%3Acreator%3E%3Cdc%3Asource%20xsi%3Atype%3D%22dcterms%3AURI%22%3Ehttps%3A%2F%2Fwww.dicebear.com%3C%2Fdc%3Asource%3E%3Cdcterms%3Alicense%20xsi%3Atype%3D%22dcterms%3AURI%22%3Ehttps%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F%3C%2Fdcterms%3Alicense%3E%3Cdc%3Arights%3E%E2%80%9CShadows%E2%80%9D%20(https%3A%2F%2Fwww.dicebear.com)%20by%20%E2%80%9CDiceBear%E2%80%9D%2C%20licensed%20under%20%E2%80%9CCC0%201.0%E2%80%9D%20(https%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F)%3C%2Fdc%3Arights%3E%3C%2Frdf%3ADescription%3E%3C%2Frdf%3ARDF%3E%3C%2Fmetadata%3E%3Cdefs%3E%3Cg%20id%3D%22shoulders-uneven-a881033c%22%3E%3Cpath%20d%3D%22M6%2050C6%2015%2020%200%2039%201c18%201%2033%2018%2033%2049Z%22%20fill%3D%22%233b3b42%22%2F%3E%3C%2Fg%3E%3Cg%20id%3D%22heads-taper-a881033c%22%3E%3Cpath%20d%3D%22M31%2019c12%200%2019%207%2019%2016%200%2012-9%2022-19%2022S12%2047%2012%2035c0-9%207-16%2019-16%22%20fill%3D%22%233b3b42%22%2F%3E%3C%2Fg%3E%3CclipPath%20id%3D%22clip-a881033c%22%3E%3Crect%20width%3D%22100%22%20height%3D%22100%22%20rx%3D%220%22%20ry%3D%220%22%2F%3E%3C%2FclipPath%3E%3C%2Fdefs%3E%3Cg%20clip-path%3D%22url(%23clip-a881033c)%22%3E%3Crect%20width%3D%22100%22%20height%3D%22100%22%20fill%3D%22%23d6eddf%22%2F%3E%3Cuse%20transform%3D%22translate(11%2051)%22%20href%3D%22%23shoulders-uneven-a881033c%22%2F%3E%3Cuse%20transform%3D%22translate(19%201)%22%20href%3D%22%23heads-taper-a881033c%22%2F%3E%3C%2Fg%3E%3C%2Fsvg%3E)
I think I found the latent direction in Gemma (an SAE feature) that represents the pandemic era... Interpreted by projecting the vector to vocabulary space, yielding a list of tokens associated with it
2 reposts12 likes
Post
You got Covid, Zoom, Fauci, Moderna, Trump/Biden, Tiktok, lockdowns, quarantines, fentanyl, the metaverse, and NFTs! Really gives you PTSD... Encountered this funny SAE feature again (MLP 22.6656), found in our research on interpreting features in LLMs: aclanthology.org/2025.acl-lo...
aclanthology.orgEnhancing Automated Interpretability with Output-Centric Feature DescriptionsYoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, Mor Geva. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.