End-to-end optimization of goal-driven and visually grounded dialogue systems

Abstract

End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architectures for sequence-to-sequence learning. Yet, most current approaches cast human-machine dialogue management as a supervised learning problem, aiming at predicting the next utterance of a participant given the full history of the dialogue. This vision is too simplistic to render the intrinsic planning problem inherent to dialogue as well as its grounded nature, making the context of a dialogue larger than the sole history. This is why only chit-chat and question answering tasks have been addressed so far using end-to-end architectures. In this paper, we introduce a Deep Reinforcement Learning method to optimize visually grounded task-oriented dialogues, based on the policy gradient algorithm. This approach is tested on a dataset of 120k dialogues collected through Mechanical Turk and provides encouraging results at solving both the problem of generating natural dialogues and the task of discovering a specific object in a complex picture.

References in corpus (6)

• A Neural Conversational Model
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• VQA: Visual Question Answering
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• An Actor-Critic Algorithm for Sequence Prediction
• PARADISE: A Framework for Evaluating Spoken Dialogue Agents

Cited by in corpus (22)

• Dialog-based Interactive Image Retrieval
• Visual Reference Resolution using Attention Memory for Visual Dialog
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
• Emergent Communication in a Multi-Modal, Multi-Step Referential Game
• A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces
• Active Learning for Visual Question Answering: An Empirical Study
• Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards
• CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
• Why Build an Assistant in Minecraft?
• Question Asking as Program Generation
• Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat
• Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game
• I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion
• Multimodal Differential Network for Visual Question Generation
• Visual Dialogue State Tracking for Question Generation
• Granular Multimodal Attention Networks for Visual Dialog
• Deep Exemplar Networks for VQA and VQG
• Dynamic Attention Networks for Task Oriented Grounding
• Reasoning Over History: Context Aware Visual Dialog
• Improving Search through A3C Reinforcement Learning based Conversational Agent
• Guessing State Tracking for Visual Dialogue