Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model

Abstract

We present a novel training framework for neural sequence models, particularly for grounded dialog generation. The standard training paradigm for these models is maximum likelihood estimation (MLE), or minimizing the cross-entropy of the human responses. Across a variety of domains, a recurring problem with MLE trained generative neural dialog models (G) is that they tend to produce 'safe' and generic responses ("I don't know", "I can't tell"). In contrast, discriminative dialog models (D) that are trained to rank a list of candidate human responses outperform their generative counterparts; in terms of automatic metrics, diversity, and informativeness of the responses. However, D is not useful in practice since it cannot be deployed to have real conversations with users. Our work aims to achieve the best of both worlds -- the practical usefulness of G and the strong performance of D -- via knowledge transfer from D to G. Our primary contribution is an end-to-end trainable generative visual dialog model, where G receives gradients from D as a perceptual (not adversarial) loss of the sequence sampled from G. We leverage the recently proposed Gumbel-Softmax (GS) approximation to the discrete distribution -- specifically, an RNN augmented with a sequence of GS samplers, coupled with the straight-through gradient estimator to enable end-to-end differentiability. We also introduce a stronger encoder for visual dialog, and employ a self-attention mechanism for answer encoding along with a metric learning loss to aid D in better capturing semantic similarities in answer responses. Overall, our proposed model outperforms state-of-the-art on the VisDial dataset by a significant margin (2.67% on recall@10). The source code can be downloaded from https://github.com/jiasenlu/visDial.pytorch.

arXiv comment11 pages, 3 figures

References in corpus (28)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Categorical Reparameterization with Gumbel-Softmax
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• Sequence Level Training with Recurrent Neural Networks
• Energy-based Generative Adversarial Network
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
• Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
• Adversarial Learning for Neural Dialogue Generation
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
• Net2Net: Accelerating Learning via Knowledge Transfer
• Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation
• Towards Diverse and Natural Image Descriptions via a Conditional GAN
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Learning End-to-End Goal-Oriented Dialog
• Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
• End-to-end optimization of goal-driven and visually grounded dialogue systems
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• GuessWhat?! Visual object discovery through multi-modal dialogue

Cited by in corpus (10)

• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• VD-BERT: A Unified Vision and Dialog Transformer with BERT
• Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Learning to Respond with Your Favorite Stickers: A Framework of Unifying Multi-Modality and User Preference in Multi-Turn Dialog
• DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue
• Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data
• ORD: Object Relationship Discovery for Visual Dialogue Generation
• Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog
• Modeling Explicit Concerning States for Reinforcement Learning in Visual Dialogue