Say What I Want: Towards the Dark Side of Neural Dialogue Models

Abstract

Neural dialogue models have been widely adopted in various chatbot applications because of their good performance in simulating and generalizing human conversations. However, there exists a dark side of these models -- due to the vulnerability of neural networks, a neural dialogue model can be manipulated by users to say what they want, which brings in concerns about the security of practical chatbot services. In this work, we investigate whether we can craft inputs that lead a well-trained black-box neural dialogue model to generate targeted outputs. We formulate this as a reinforcement learning (RL) problem and train a Reverse Dialogue Generator which efficiently finds such inputs for targeted outputs. Experiments conducted on a representative neural dialogue model show that our proposed model is able to discover such desired inputs in a considerable portion of cases. Overall, our work reveals this weakness of neural dialogue models and may prompt further researches of developing corresponding solutions to avoid it.

arXiv comment11 pages, 2 figures

References in corpus (8)

• A Neural Conversational Model
• Deep Text Classification Can be Fooled
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
• Adversarial Attacks on Neural Network Policies
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• Adversarial Images for Variational Autoencoders
• Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer

Cited by in corpus (5)

• Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
• Does Gender Matter? Towards Fairness in Dialogue Systems
• Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning
• Learning Multi-level Dependencies for Robust Word Recognition
• Automatically Exposing Problems with Neural Dialog Models