Making Sense Of Distributed Representations With Activation Spectroscopy

Kyle Reing, Greg Ver Steeg, Aram Galstyan

In the study of neural network interpretability, there is growing evidence to suggest that relevant features are encoded across many neurons in a distributed fashion. Making sense…