good thread by someone smarter than me explaining that AI interpretability explainer:
ok having read it, let me try. Background knowledge: a neural network, conceptually, has billions of little things called neurons. One goal of so-called "interpretability" would be to be able to look at each neuron and say what it does for the network.