34 citations · 75 across the 13 of their papers we have counts for
13 papers
Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?
Wen Shen, Lei Cheng, Yuxiao Yang +2
In this paper, we explain the inference logic of large language models (LLMs) as a set of symbolic concepts. Many recent studies have discovered that traditional DNNs usually encod…
Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNs
Mingjie Li, Quanshi Zhang
In this technical note, we aim to explain a deep neural network (DNN) by quantifying the encoded interactions between input variables, which reflects the DNN's inference logic. Spe…
Does a Neural Network Really Encode Symbolic Concepts?
Mingjie Li, Quanshi Zhang
Recently, a series of studies have tried to extract interactions between input variables modeled by a DNN and define such interactions as concepts encoded by the DNN. However, stri…
SymNMF-Net for The Symmetric NMF Problem
Mingjie Li, Hao Kong, Zhouchen Lin
Recently, many works have demonstrated that Symmetric Non-negative Matrix Factorization~(SymNMF) enjoys a great superiority for various clustering tasks. Although the state-of-the-…
Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs
Jie Ren, Mingjie Li, Meng Zhou +2
This paper aims to theoretically analyze the complexity of feature transformations encoded in piecewise linear DNNs with ReLU layers. We propose metrics to measure three types of c…
Visualizing the Emergence of Intermediate Visual Patterns in DNNs
Mingjie Li, Shaobo Wang, Quanshi Zhang
This paper proposes a method to visualize the discrimination power of intermediate-layer visual patterns encoded by a DNN. Specifically, we visualize (1) how the DNN gradually lear…
Defining and Quantifying the Emergence of Sparse Concepts in DNNs
Jie Ren, Mingjie Li, Qirui Chen +2
This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a fe…
Pareto Adversarial Robustness: Balancing Spatial Robustness and Sensitivity-based Robustness
Ke Sun, Mingjie Li, Zhouchen Lin
Adversarial robustness, which primarily comprises sensitivity-based robustness and spatial robustness, plays an integral part in achieving robust generalization. In this paper, we…
Generating Watermarked Adversarial Texts
Mingjie Li, Hanzhou Wu, Xinpeng Zhang
Adversarial example generation has been a hot spot in recent years because it can cause deep neural networks (DNNs) to misclassify the generated adversarial examples, which reveals…
Interpreting and Boosting Dropout from a Game-Theoretic View
Hao Zhang, Sen Li, Yinchao Ma +3
This paper aims to understand and improve the utility of the dropout operation from the perspective of game-theoretic interactions. We prove that dropout can suppress the strength…
Interpreting and Disentangling Feature Components of Various Complexity from DNNs
Jie Ren, Mingjie Li, Zexu Liu +1
This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a…
IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
Ke Sun, Mingjie Li, Dong Liu +1
In this paper, we are interested in building lightweight and efficient convolutional neural networks. Inspired by the success of two design patterns, composition of structured spar…
Deep Convolutional Neural Networks with Merge-and-Run Mappings
Liming Zhao, Jingdong Wang, Xi Li +2
A deep residual network, built by stacking a sequence of residual blocks, is easy to train, because identity mappings skip residual branches and thus improve information flow. To f…