Visual Dialogue without Vision or Dialogue
arXiv:1812.06417
Abstract
We characterise some of the quirks and shortcomings in the exploration of Visual Dialogue - a sequential question-answering task where the questions and corresponding answers are related through given visual stimuli. To do so, we develop an embarrassingly simple method based on Canonical Correlation Analysis (CCA) that, on the standard dataset, achieves near state-of-the-art performance on mean rank (MR). In direct contrast to current complex and over-parametrised architectures that are both compute and time intensive, our method ignores the visual stimuli, ignores the sequencing of dialogue, does not need gradients, uses off-the-shelf feature extractors, has at least an order of magnitude fewer parameters, and learns in practically no time. We argue that these results are indicative of issues in current approaches to Visual Dialogue and conduct analyses to highlight implicit dataset biases and effects of over-constrained evaluation metrics. Our code is publicly available.
2018 NeurIPS Workshop on Critiquing and Correcting Trends in Machine Learning
References in corpus (5)
- Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
- Blindfold Baselines for Embodied QA
- Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
- FlipDial: A Generative Model for Two-Way Visual Dialogue
- Response to "Visual Dialogue without Vision or Dialogue" (Massiceti et al., 2018)
Cited by in corpus (8)
- Multimodal Variational Autoencoders for Semi-Supervised Learning: In Defense of Product-of-Experts
- Misleading Failures of Partial-input Baselines
- Modality-Balanced Models for Visual Dialogue
- Challenges and Prospects in Vision and Language Research
- Response to "Visual Dialogue without Vision or Dialogue" (Massiceti et al., 2018)
- Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments
- A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions
- A Revised Generative Evaluation of Visual Dialogue