DCA: Diversified Co-Attention towards Informative Live Video Commenting
arXiv:1911.02739
Abstract
We focus on the task of Automatic Live Video Commenting (ALVC), which aims to generate real-time video comments with both video frames and other viewers' comments as inputs. A major challenge in this task is how to properly leverage the rich and diverse information carried by video and text. In this paper, we aim to collect diversified information from video and text for informative comment generation. To achieve this, we propose a Diversified Co-Attention (DCA) model for this task. Our model builds bidirectional interactions between video frames and surrounding comments from multiple perspectives via metric learning, to collect a diversified and informative context for comment generation. We also propose an effective parameter orthogonalization technique to avoid excessive overlap of information learned from different perspectives. Results show that our approach outperforms existing methods in the ALVC task, achieving new state-of-the-art results.
References in corpus (12)
- Bidirectional Attention Flow for Machine Comprehension
- A Structured Self-attentive Sentence Embedding
- Dynamic Coattention Networks For Question Answering
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Sequence to Sequence -- Video to Text
- Parseval Networks: Improving Robustness to Adversarial Examples
- Proactive Human-Machine Conversation with Explicit Conversation Goals
- Weakly Supervised Dense Video Captioning
- Automatic Article Commenting: the Task and Dataset
- Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
- Unsupervised Machine Commenting with Neural Variational Topic Model
- Move Forward and Tell: A Progressive Generator of Video Descriptions