Conditional Generation and Snapshot Learning in Neural Dialogue Systems
arXiv:1606.03352
Abstract
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential signals by applying a companion cross-entropy objective function to the conditioning vector. The experimental and analytical results demonstrate firstly that competition occurs between the conditioning vector and the LM, and the differing architectures provide different trade-offs between the two. Secondly, the discriminative power and transparency of the conditioning vector is key to providing both model interpretability and better performance. Thirdly, snapshot learning leads to consistent performance improvements independent of which architecture is used.
References in corpus (6)
- Sequence to Sequence Learning with Neural Networks
- Teaching Machines to Read and Comprehend
- A Neural Conversational Model
- The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
- Neural Responding Machine for Short-Text Conversation
- Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems