Why Do Neural Response Generation Models Prefer Universal Replies?
arXiv:1808.09187
Abstract
Recent advances in sequence-to-sequence learning reveal a purely data-driven approach to the response generation task. Despite its diverse applications, existing neural models are prone to producing short and generic replies, making it infeasible to tackle open-domain challenges. In this research, we analyze this critical issue in light of the model's optimization goal and the specific characteristics of the human-to-human dialog corpus. By decomposing the black box into parts, a detailed analysis of the probability limit was conducted to reveal the reason behind these universal replies. Based on these analyses, we propose a max-margin ranking regularization term to avoid the models leaning to these replies. Finally, empirical experiments on case studies and benchmarks with several metrics validate this approach.
arXiv commentPreprint of the paper presented to the AAAI 2020 Workshop on Interactive and Conversational Recommendation Systems (WICRS)
References in corpus (15)
- Sequence to Sequence Learning with Neural Networks
- A Neural Conversational Model
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Topic Aware Neural Response Generation
- A Diversity-Promoting Objective Function for Neural Conversation Models
- A Simple, Fast Diverse Decoding Algorithm for Neural Generation
- Neural Responding Machine for Short-Text Conversation
- A Survey of Available Corpora for Building Data-Driven Dialogue Systems
- A Persona-Based Neural Conversation Model
- Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
- Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation
- A Conditional Variational Framework for Dialog Generation
- Steering Output Style and Topic in Neural Response Generation
- Neural Machine Translation with Word Predictions
- Detecting Context Dependent Messages in a Conversational Environment