Task-oriented Dialogue Systems: performance vs. quality-optima, a review
arXiv:2112.11176 · doi:10.5121/csit.2022.121306
Abstract
Task-oriented dialogue systems (TODS) are continuing to rise in popularity as various industries find ways to effectively harness their capabilities, saving both time and money. However, even state-of-the-art TODS are not yet reaching their full potential. TODS typically have a primary design focus on completing the task at hand, so the metric of task-resolution should take priority. Other conversational quality attributes that may point to the success, or otherwise, of the dialogue, may be ignored. This can cause interactions between human and dialogue system that leave the user dissatisfied or frustrated. This paper explores the literature on evaluative frameworks of dialogue systems and the role of conversational quality attributes in dialogue systems, looking at if, how, and where they are utilised, and examining their correlation with the performance of the dialogue system.
References in corpus (8)
- Assessing agreement on classification tasks: the kappa statistic
- A Neural Conversational Model
- PARADISE: A Framework for Evaluating Spoken Dialogue Agents
- A Network-based End-to-End Trainable Task-oriented Dialogue System
- Metrics for Evaluating Dialogue Strategies in a Spoken Language System
- Flexible End-to-End Dialogue System for Knowledge Grounded Conversation
- A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
- Multi-domain Neural Network Language Generation for Spoken Dialogue Systems