Interpreting and Disentangling Feature Components of Various Complexity from DNNs
arXiv:2006.15920
Abstract
This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a certain layer in the DNN, our method disentangles feature components of different complexity orders from the feature. We further design a set of metrics to evaluate the reliability, the effectiveness, and the significance of over-fitting of these feature components. Furthermore, we successfully discover a close relationship between the feature complexity and the performance of DNNs. As a generic mathematical tool, the feature complexity and the proposed metrics can also be used to analyze the success of network compression and knowledge distillation.
References in corpus (21)
- Distilling the Knowledge in a Neural Network
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Understanding Neural Networks Through Deep Visualization
- Opening the Black Box of Deep Neural Networks via Information
- Similarity of Neural Network Representations Revisited
- Learning how to explain neural networks: PatternNet and PatternAttribution
- Sensitivity and Generalization in Neural Networks: an Empirical Study
- How to Construct Deep Recurrent Neural Networks
- Information-theoretic analysis of generalization capability of learning algorithms
- Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
- Understanding Deep Neural Networks with Rectified Linear Units
- The power of deeper networks for expressing natural functions
- Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
- On the Computational Efficiency of Training Neural Networks
- Stiffness: A New Perspective on Generalization in Neural Networks
- Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
- Understanding training and generalization in deep learning by Fourier analysis
- Knowledge Consistency between Neural Networks and Beyond
- The Computational Complexity of Training ReLU(s)
- Complexity of Training ReLU Neural Network