33 citations · 439 across the 68 of their papers we have counts for
68 papers
Explaining How a Neural Network Play the Go Game and Let People Learn
Huilin Zhou, Huijie Tang, Mingjie Li +3
The AI model has surpassed human players in the game of Go, and it is widely believed that the AI model has encoded new knowledge about the Go game beyond human players. In this wa…
Towards Attributions of Input Variables in a Coalition
Xinhao Zheng, Huiqi Deng, Quanshi Zhang
This paper focuses on the fundamental challenge of partitioning input variables in attribution methods for Explainable AI, particularly in Shapley value-based approaches. Previous…
Where We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Models
Qihan Ren, Jiayang Gao, Wen Shen +1
This study aims to prove the emergence of symbolic concepts (or more precisely, sparse primitive inference patterns) in well-trained deep neural networks (DNNs). Specifically, we p…
Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?
Wen Shen, Lei Cheng, Yuxiao Yang +2
In this paper, we explain the inference logic of large language models (LLMs) as a set of symbolic concepts. Many recent studies have discovered that traditional DNNs usually encod…
Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNs
Mingjie Li, Quanshi Zhang
In this technical note, we aim to explain a deep neural network (DNN) by quantifying the encoded interactions between input variables, which reflects the DNN's inference logic. Spe…
HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation
Lu Chen, Siyu Lou, Keyan Zhang +2
The Shapley value is widely regarded as a trustworthy attribution metric. However, when people use Shapley values to explain the attribution of input variables of a deep neural net…
Understanding and Unifying Fourteen Attribution Methods with Taylor Interactions
Huiqi Deng, Na Zou, Mengnan Du +5
Various attribution methods have been developed to explain deep neural networks (DNNs) by inferring the attribution/importance/contribution score of each input variable to the fina…
Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts
Qihan Ren, Huiqi Deng, Yunuo Chen +2
In this paper, we focus on mean-field variational Bayesian Neural Networks (BNNs) and explore the representation capacity of such BNNs by investigating which types of concepts are…
Explaining Generalization Power of a DNN Using Interactive Concepts
Huilin Zhou, Hao Zhang, Huiqi Deng +4
This paper explains the generalization power of a deep neural network (DNN) from the perspective of interactions. Although there is no universally accepted definition of the concep…
Does a Neural Network Really Encode Symbolic Concepts?
Mingjie Li, Quanshi Zhang
Recently, a series of studies have tried to extract interactions between input variables modeled by a DNN and define such interactions as concepts encoded by the DNN. However, stri…
Defects of Convolutional Decoder Networks in Frequency Representation
Ling Tang, Wen Shen, Zhanpeng Zhou +2
In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input…
Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification
Quanshi Zhang, Xu Cheng, Yilan Chen +1
Compared to traditional learning from scratch, knowledge distillation sometimes makes the DNN achieve superior performance. This paper provides a new perspective to explain the suc…
Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferability
Quanshi Zhang, Xin Wang, Jie Ren +4
Although many methods have been proposed to enhance the transferability of adversarial perturbations, these methods are designed in a heuristic manner, and the essential mechanism…
Batch Normalization Is Blind to the First and Second Derivatives of the Loss
Zhanpeng Zhou, Wen Shen, Huixin Chen +2
In this paper, we prove the effects of the BN operation on the back-propagation of the first and second derivatives of the loss. When we do the Taylor series expansion of the loss…
Why Adversarial Training of ReLU Networks Is Difficult?
Xu Cheng, Hao Zhang, Yue Xin +3
This paper mathematically derives an analytic solution of the adversarial perturbation on a ReLU network, and theoretically explains the difficulty of adversarial training. Specifi…
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Jiexing Qi, Jingyao Tang, Ziwei He +6
Relational structures such as schema linking and schema encoding have been validated as a key component to qualitatively translating natural language into SQL queries. However, int…
Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs
Jie Ren, Mingjie Li, Meng Zhou +2
This paper aims to theoretically analyze the complexity of feature transformations encoded in piecewise linear DNNs with ReLU layers. We propose metrics to measure three types of c…
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Trap of Feature Diversity in the Learning of MLPs
Dongrui Liu, Shaobo Wang, Jie Ren +4
In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs), and we aim to explain the reason for the decrease of feature diversity…
A Unified Game-Theoretic Interpretation of Adversarial Robustness
Jie Ren, Die Zhang, Yisen Wang +8
This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i.e.} the view of multi-order interactions between input variables of DNNs. B…
Interpreting Representation Quality of DNNs for 3D Point Cloud Processing
Wen Shen, Qihan Ren, Dongrui Liu +1
In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the ove…
Visualizing the Emergence of Intermediate Visual Patterns in DNNs
Mingjie Li, Shaobo Wang, Quanshi Zhang
This paper proposes a method to visualize the discrimination power of intermediate-layer visual patterns encoded by a DNN. Specifically, we visualize (1) how the DNN gradually lear…
Discovering and Explaining the Representation Bottleneck of DNNs
Huiqi Deng, Qihan Ren, Hao Zhang +1
This paper explores the bottleneck of feature representations of deep neural networks (DNNs), from the perspective of the complexity of interactions between input variables encoded…
Defining and Quantifying the Emergence of Sparse Concepts in DNNs
Jie Ren, Mingjie Li, Qirui Chen +2
This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a fe…