Post

Victor Geislinger
@victorsvector.com

Thanks for sharing your blog post! But I also would so be up for a long thread form of this XD When you mentioned working in the latent space, I instantly thought of this late 2023 paper by Google Research arxiv.org/abs/2310.04475

arxiv.orgDemystifying Embedding Spaces using Large Language ModelsEmbeddings have become a pivotal means to represent complex, multi-faceted information about entities, concepts, and relationships in a condensed and useful format. Nevertheless, they often preclude d...
mr. TIM
@timkellogg.me

interesting, i’ve heard of CAVs, but they always seemed more of a gray box approach. it seems like you could train a model to translate an embedding back into text. doesn’t seem like it would be too hard. also seems inevitable if we go all in on latent space

Victor Geislinger
@victorsvector.com

I believe the latent space almost always means that there's potential for more accessible information than just text with the space being continuously differentiable. I think of early days variational autoencoders that could be used to generate beyond training data

Victor Geislinger
@victorsvector.com

But translating embedding back to text seems to me not necessarily straightforward. Where I've seen success is when comparisons are done with a 'well-understood' embedding. I can see conversion to text being difficult in a similar way that there doesn't exist perfect translation between languages

0 reposts1 like