A Game-Theoretic Taxonomy of Visual Concepts in DNNs
arXiv:2106.10938
Abstract
In this paper, we rethink how a DNN encodes visual concepts of different complexities from a new perspective, i.e. the game-theoretic multi-order interactions between pixels in an image. Beyond the categorical taxonomy of objects and the cognitive taxonomy of textures and shapes, we provide a new taxonomy of visual concepts, which helps us interpret the encoding of shapes and textures, in terms of concept complexities. In this way, based on multi-order interactions, we find three distinctive signal-processing behaviors of DNNs encoding textures. Besides, we also discover the flexibility for a DNN to encode shapes is lower than the flexibility of encoding textures. Furthermore, we analyze how DNNs encode outlier samples, and explore the impacts of network architectures on interactions. Additionally, we clarify the crucial role of the multi-order interactions in real-world applications. The code will be released when the paper is accepted.
arXiv comment12 pages
References in corpus (12)
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Improved Regularization of Convolutional Neural Networks with Cutout
- Understanding Neural Networks Through Deep Visualization
- Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
- A Benchmark for Interpretability Methods in Deep Neural Networks
- Learning how to explain neural networks: PatternNet and PatternAttribution
- Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
- Visualizing and Understanding Atari Agents
- Explanations can be manipulated and geometry is to blame
- Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
- XRAI: Better Attributions Through Regions
- A Singular Value Perspective on Model Robustness