Automatically Exposing Problems with Neural Dialog Models
arXiv:2109.06950
Abstract
Neural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by model designers through interactions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. In this paper, we propose two methods including reinforcement learning to automatically trigger a dialog model into generating problematic responses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models.
References in corpus (13)
- Decoupled Weight Decay Regularization
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Categorical Reparameterization with Gumbel-Softmax
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- End-To-End Memory Networks
- Fine-Tuning Language Models from Human Preferences
- Learning to Generate Reviews and Discovering Sentiment
- A Deep Reinforcement Learning Chatbot
- Recipes for Safety in Open-domain Chatbots
- Detecting egregious responses in neural sequence-to-sequence models
- Say What I Want: Towards the Dark Side of Neural Dialogue Models
- Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models