10 citations · 33 across the 10 of their papers we have counts for
6 papers · 1 filter
The Interaction Bottleneck of Deep Neural Networks: Discovery, Proof, and Modulation
Huiqi Deng, Qihan Ren, Zhuofan Chen +5
Understanding what kinds of cooperative structures deep neural networks (DNNs) can represent remains a fundamental yet insufficiently understood problem. In this work, we treat int…
Where We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Models
Qihan Ren, Jiayang Gao, Wen Shen +1
This study aims to prove the emergence of symbolic concepts (or more precisely, sparse primitive inference patterns) in well-trained deep neural networks (DNNs). Specifically, we p…
Explaining Generalization Power of a DNN Using Interactive Concepts
Huilin Zhou, Hao Zhang, Huiqi Deng +4
This paper explains the generalization power of a deep neural network (DNN) from the perspective of interactions. Although there is no universally accepted definition of the concep…
Defects of Convolutional Decoder Networks in Frequency Representation
Ling Tang, Wen Shen, Zhanpeng Zhou +2
In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input…
Batch Normalization Is Blind to the First and Second Derivatives of the Loss
Zhanpeng Zhou, Wen Shen, Huixin Chen +2
In this paper, we prove the effects of the BN operation on the back-propagation of the first and second derivatives of the loss. When we do the Taylor series expansion of the loss…
Why Adversarial Training of ReLU Networks Is Difficult?
Xu Cheng, Hao Zhang, Yue Xin +3
This paper mathematically derives an analytic solution of the adversarial perturbation on a ReLU network, and theoretically explains the difficulty of adversarial training. Specifi…