HUMBO: Bridging Response Generation and Facial Expression Synthesis
arXiv:1905.11240
Abstract
Spoken dialogue systems that assist users to solve complex tasks such as movie ticket booking have become an emerging research topic in artificial intelligence and natural language processing areas. With a well-designed dialogue system as an intelligent personal assistant, people can accomplish certain tasks more easily via natural language interactions. Today there are several virtual intelligent assistants in the market; however, most systems only focus on textual or vocal interaction. In this paper, we present HUMBO, a system aiming at generating dialogue responses and simultaneously synthesize corresponding visual expressions on faces for better multimodal interaction. HUMBO can (1) let users determine the appearances of virtual assistants by a single image, and (2) generate coherent emotional utterances and facial expressions on the user-provided image. This is not only a brand new research direction but more importantly, an ultimate step toward more human-like virtual assistants.
arXiv commentThe first two authors contributed to this work equally
References in corpus (8)
- Sequence to Sequence Learning with Neural Networks
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- VQA: Visual Question Answering
- Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks
- EmotionLines: An Emotion Corpus of Multi-Party Conversations
- Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
- Investigating Linguistic Pattern Ordering in Hierarchical Natural Language Generation