Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7

Abstract

Scene-aware dialog systems will be able to have conversations with users about the objects and events around them. Progress on such systems can be made by integrating state-of-the-art technologies from multiple research areas including end-to-end dialog systems visual dialog, and video description. We introduce the Audio Visual Scene Aware Dialog (AVSD) challenge and dataset. In this challenge, which is one track of the 7th Dialog System Technology Challenges (DSTC7) workshop1, the task is to build a system that generates responses in a dialog about an input video

References in corpus (3)

• VQA: Visual Question Answering
• Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

Cited by in corpus (9)

• Dialog System Technology Challenge 7
• From FiLM to Video: Multi-turn Question Answering with Multi-modal Context
• DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator
• Factor Graph Attention
• Reactive Multi-Stage Feature Fusion for Multimodal Dialogue Modeling
• End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features
• VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
• Leveraging Topics and Audio Features with Multimodal Attention for Audio Visual Scene-Aware Dialog
• Exploring Context, Attention and Audio Features for Audio Visual Scene-Aware Dialog