Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System
arXiv:2008.01508
Abstract
Recently, there has been increasing interest in transparency and interpretability in Deep Reinforcement Learning (DRL) systems. Verbal explanations, as the most natural way of communication in our daily life, deserve more attention, since they allow users to gain a better understanding of the system which ultimately could lead to a high level of trust and smooth collaboration. This paper reports a novel work in generating verbal explanations for DRL behaviors agent. A rule-based model is designed to construct explanations using a series of rules which are predefined with prior knowledge. A learning model is then proposed to expand the implicit logic of generating verbal explanation to general situations by employing rule-based explanations as training data. The learning model is shown to have better flexibility and generalizability than the static rule-based model. The performance of both models is evaluated quantitatively through objective metrics. The results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems. Additionally, seven variants of the learning model are designed to illustrate the contribution of input channels, attention mechanism, and proposed encoder in improving the quality of verbal explanation.
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Striving for Simplicity: The All Convolutional Net
- Understanding Black-box Predictions via Influence Functions
- Recurrent Models of Visual Attention
- Learning to Navigate in Complex Environments
- Visualizing and Understanding Atari Agents
- Playing FPS Games with Deep Reinforcement Learning
- Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
- Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations