The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms

Adam Davies, Ashkan Khakzar

Artificial neural networks have long been understood as "black boxes": though we know their computation graphs and learned parameters, the knowledge encoded by these weights and fu…