Analyzing Representations inside Convolutional Neural Networks
arXiv:2012.12516
Abstract
How can we discover and succinctly summarize the concepts that a neural network has learned? Such a task is of great importance in applications of networks in areas of inference that involve classification, like medical diagnosis based on fMRI/x-ray etc. In this work, we propose a framework to categorize the concepts a network learns based on the way it clusters a set of input examples, clusters neurons based on the examples they activate for, and input features all in the same latent space. This framework is unsupervised and can work without any labels for input features, it only needs access to internal activations of the network for each input example, thereby making it widely applicable. We extensively evaluate the proposed method and demonstrate that it produces human-understandable and coherent concepts that a ResNet-18 has learned on the CIFAR-100 dataset.
arXiv commentWork in Progress
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- End to End Learning for Self-Driving Cars
- SmoothGrad: removing noise by adding noise
- Visualizing and Understanding Convolutional Networks
- Towards Robust Interpretability with Self-Explaining Neural Networks
- SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
- Understanding Deep Image Representations by Inverting Them
- Deep Manifold Traversal: Changing Labels with Convolutional Features