Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog

Abstract

Creating an intelligent conversational system that understands vision and language is one of the ultimate goals in Artificial Intelligence (AI)~\cite{winograd1972understanding}. Extensive research has focused on vision-to-language generation, however, limited research has touched on combining these two modalities in a goal-driven dialog context. We propose a multimodal hierarchical reinforcement learning framework that dynamically integrates vision and language for task-oriented visual dialog. The framework jointly learns the multimodal dialog state representation and the hierarchical dialog policy to improve both dialog task success and efficiency. We also propose a new technique, state adaptation, to integrate context awareness in the dialog state representation. We evaluate the proposed framework and the state adaptation technique in an image guessing game and achieve promising results.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Prioritized Experience Replay
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• VQA: Visual Question Answering
• Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
• Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation
• Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• Deep Reinforcement Learning with a Natural Language Action Space
• Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards

Cited by in corpus (3)

• Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
• Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
• Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation