Interpretability Illusions in the Generalization of Simplified Models

Dan Friedman, Andrew Lampinen, Lucas Dixon +2

A common method to study deep learning systems is to use simplified model representations--for example, using singular value decomposition to visualize the model's hidden states in…