Post

Are similar representations in neural nets evidence of shared computation? In new theory work w/ Lukas Braun (lukasbraun.com) & @saxelab.bsky.social, we prove that representational comparisons are ill-posed in general, unless networks are efficient. @icmlconf.bsky.social @cogcompneuro.bsky.social

To analyse this dissociation in a tractable model of representation learning, we characterize *all* task solutions for two-layer linear networks. Within this solution manifold, we identify a solution hierarchy in terms of what implicit objectives are minimized (in addition to the task objective).

We parametrised this solution hierarchy to find differences in handling of task-irrelevant dimensions: Some solutions compress away (creating task-specific, interpretable representations), while others preserve arbitrary structure in null spaces (creating arbitrary, uninterpretable representations).

We demonstrate that representation analysis and comparison is ill-posed, giving both false negatives and false positives, unless we work with *task-specific representations*. These are interpretable *and* robust to noise (i.e., representational identifiability comes with computational advantages).