Local Explanation of Dialogue Response Generation

Abstract

In comparison to the interpretation of classification models, the explanation of sequence generation models is also an important problem, however it has seen little attention. In this work, we study model-agnostic explanations of a representative text generation task -- dialogue response generation. Dialog response generation is challenging with its open-ended sentences and multiple acceptable responses. To gain insights into the reasoning process of a generation model, we propose a new method, local explanation of response generation (LERG) that regards the explanations as the mutual interaction of segments in input and output sentences. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. We show that LERG adheres to desired properties of explanations for text generation including unbiased approximation, consistency and cause identification. Empirically, our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%. Our analysis demonstrates that LERG can extract both explicit and implicit relations between input and output segments.

arXiv commentaccepted to NeurIPS 2021

References in corpus (9)

• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• A Neural Conversational Model
• Towards Robust Interpretability with Self-Explaining Neural Networks
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
• Debiasing Concept-based Explanations with Causal Analysis
• Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks