20 citations · 68 across the 17 of their papers we have counts for
18 papers
Faithful Activation Verbalization: Reducing Hallucinations in LLM Representation Interpretation
Haiyan Zhao, Zirui Hei, Wei Shi +3
Activation verbalization methods such as Activation Oracle and Natural Language Autoencoders decode hidden representations of large language models into human-readable natural lang…
Do LLMs Know Their Vulnerable Scenarios?
Ziheng Peng, Huiqi Deng, Haoran Jing +5
Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards. Existing red-teami…
Your LLM, Your Style: Behavioral Mode Axes for LLM Behavioral Control
Haoze Liu, Run Liu, Haiying Xu +6
Large language models (LLMs) increasingly act in interactive settings where their behavioral styles affect user experience, safety, and downstream decision making. Existing LLM per…
SkillEval: Decomposing Agent Skill Quality into Interpretable Signals
Jiahui Han, Qinuo Li, Ziheng Peng +6
Agent skills provide reusable knowledge that helps agents solve specialized tasks. As their use expands, evaluating skill quality becomes increasingly important. Existing evaluatio…
AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
Dongrui Liu, Qihan Ren, Chen Qian +40
The rise of AI agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current guardrail models lack agentic risk…
Attribution Explanations for Deep Neural Networks: A Theoretical Perspective
Huiqi Deng, Hongbin Pei, Quanshi Zhang +1
Attribution explanation is a typical approach for explaining deep neural networks (DNNs), inferring an importance or contribution score for each input variable to the final output.…
Mitigating Shortcuts in Language Models with Soft Label Encoding
Zirui He, Huiqi Deng, Haiyan Zhao +2
Recent research has shown that large language models rely on spurious correlations in the data for natural language understanding (NLU) tasks. In this work, we aim to answer the fo…
Towards Attributions of Input Variables in a Coalition
Xinhao Zheng, Huiqi Deng, Quanshi Zhang
This paper focuses on the fundamental challenge of partitioning input variables in attribution methods for Explainable AI, particularly in Shapley value-based approaches. Previous…
Explainability for Large Language Models: A Survey
Haiyan Zhao, Hanjie Chen, Fan Yang +6
Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transpa…
Understanding and Unifying Fourteen Attribution Methods with Taylor Interactions
Huiqi Deng, Na Zou, Mengnan Du +5
Various attribution methods have been developed to explain deep neural networks (DNNs) by inferring the attribution/importance/contribution score of each input variable to the fina…
Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts
Qihan Ren, Huiqi Deng, Yunuo Chen +2
In this paper, we focus on mean-field variational Bayesian Neural Networks (BNNs) and explore the representation capacity of such BNNs by investigating which types of concepts are…
Explaining Generalization Power of a DNN Using Interactive Concepts
Huilin Zhou, Hao Zhang, Huiqi Deng +4
This paper explains the generalization power of a deep neural network (DNN) from the perspective of interactions. Although there is no universally accepted definition of the concep…
Trap of Feature Diversity in the Learning of MLPs
Dongrui Liu, Shaobo Wang, Jie Ren +4
In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs), and we aim to explain the reason for the decrease of feature diversity…
Discovering and Explaining the Representation Bottleneck of DNNs
Huiqi Deng, Qihan Ren, Hao Zhang +1
This paper explores the bottleneck of feature representations of deep neural networks (DNNs), from the perspective of the complexity of interactions between input variables encoded…
Defining and Quantifying the Emergence of Sparse Concepts in DNNs
Jie Ren, Mingjie Li, Qirui Chen +2
This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a fe…
A General Taylor Framework for Unifying and Revisiting Attribution Methods
Huiqi Deng, Na Zou, Mengnan Du +3
Attribution methods provide an insight into the decision-making process of machine learning models, especially deep neural networks, by assigning contribution scores to each indivi…
Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attribution
Huiqi Deng, Na Zou, Weifu Chen +3
Back propagation based visualizations have been proposed to interpret deep neural networks (DNNs), some of which produce interpretations with good visual quality. However, there ex…
A Unified Taylor Framework for Revisiting Attribution Methods
Huiqi Deng, Na Zou, Mengnan Du +3
Attribution methods have been developed to understand the decision-making process of machine learning models, especially deep neural networks, by assigning importance scores to ind…