Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems

Abstract

In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommunication to demonstrate what a human-like interactive experience should be and how to achieve smooth turn-taking through three subtasks: user state detection, backchannel selection, and barge-in detection. Besides, we propose semi-supervised learning with multimodal data augmentation to leverage unlabeled data to increase model generalization. Experimental results on three sub-tasks show that the proposed method achieves consistent improvements compared with baselines. We deploy the Duplex Conversation to Alibaba intelligent customer service and share lessons learned in production. Online A/B experiments show that the proposed system can significantly reduce response latency by 50%.

Accepted by KDD 2022, ADS track

References in corpus (8)

• Spoken Language Interaction with Robots: Research Issues and Recommendations, Report from the NSF Future Directions Workshop
• Deep Open Intent Classification with Adaptive Decision Boundary
• Discovering New Intents with Deep Aligned Clustering
• GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection
• TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog
• TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition
• EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
• A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slots