Discovering and Explaining the Representation Bottleneck of DNNs

Abstract

This paper explores the bottleneck of feature representations of deep neural networks (DNNs), from the perspective of the complexity of interactions between input variables encoded in DNNs. To this end, we focus on the multi-order interaction between input variables, where the order represents the complexity of interactions. We discover that a DNN is more likely to encode both too simple interactions and too complex interactions, but usually fails to learn interactions of intermediate complexity. Such a phenomenon is widely shared by different DNNs for different tasks. This phenomenon indicates a cognition gap between DNNs and human beings, and we call it a representation bottleneck. We theoretically prove the underlying reason for the representation bottleneck. Furthermore, we propose a loss to encourage/penalize the learning of interactions of specific complexities, and analyze the representation capacities of interactions of different complexities.

References in corpus (14)

• Axiomatic Attribution for Deep Networks
• Learning deep representations by mutual information estimation and maximization
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• On the Number of Linear Regions of Deep Neural Networks
• Opening the Black Box of Deep Neural Networks via Information
• Consistent Individualized Feature Attribution for Tree Ensembles
• A Closer Look at Memorization in Deep Networks
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• On the number of response regions of deep feed forward networks with piece-wise linear activations
• Stiffness: A New Perspective on Generalization in Neural Networks
• Understanding training and generalization in deep learning by Fourier analysis
• Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection