Post

Are similar representations in neural nets evidence of shared computation? In new theory work w/ Lukas Braun (lukasbraun.com) & @saxelab.bsky.social, we prove that representational comparisons are ill-posed in general, unless networks are efficient. @icmlconf.bsky.social @cogcompneuro.bsky.social

Deep networks have parameter symmetries, so we can walk through solution space, changing all weights and representations, while keeping output fixed. In the worst case, function and representation are *dissociated*. (Networks can have the same function with the same or different representation.)

We parametrised this solution hierarchy to find differences in handling of task-irrelevant dimensions: Some solutions compress away (creating task-specific, interpretable representations), while others preserve arbitrary structure in null spaces (creating arbitrary, uninterpretable representations).

We demonstrate that representation analysis and comparison is ill-posed, giving both false negatives and false positives, unless we work with *task-specific representations*. These are interpretable *and* robust to noise (i.e., representational identifiability comes with computational advantages).

Function-representation dissociations and the representation-computation link persist in deep nonlinear networks! Using function-invariant reparametrisations (@bsimsek.bsky.social), we break representational identifiability but degrade generalization (a computational consequence).