Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems

Abstract

Self-attentional models are a new paradigm for sequence modelling tasks which differ from common sequence modelling methods, such as recurrence-based and convolution-based sequence learning, in the way that their architecture is only based on the attention mechanism. Self-attentional models have been used in the creation of the state-of-the-art models in many NLP tasks such as neural machine translation, but their usage has not been explored for the task of training end-to-end task-oriented dialogue generation systems yet. In this study, we apply these models on the three different datasets for training task-oriented chatbots. Our finding shows that self-attentional models can be exploited to create end-to-end task-oriented chatbots which not only achieve higher evaluation scores compared to recurrence-based models, but also do so more efficiently.

arXiv commentAppeared in proceedings of Recent Advances in Natural Language Processing (RANLP) Conference, 2019

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• WaveNet: A Generative Model for Raw Audio
• Convolutional Sequence to Sequence Learning
• Neural Machine Translation in Linear Time
• Conversational AI: The Science Behind the Alexa Prize
• A Network-based End-to-End Trainable Task-oriented Dialogue System
• Building a Conversational Agent Overnight with Dialogue Self-Play
• End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning
• Generating Wikipedia by Summarizing Long Sequences
• Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems