Image to Video Domain Adaptation Using Web Supervision

Abstract

Training deep neural networks typically requires large amounts of labeled data which may be scarce or expensive to obtain for a particular target domain. As an alternative, we can leverage webly-supervised data (i.e. results from a public search engine) which are relatively plentiful but may contain noisy results. In this work, we propose a novel two-stage approach to learn a video classifier using webly-supervised data. We argue that learning appearance features and then temporal features sequentially, rather than simultaneously, is an easier optimization for this task. We show this by first learning an image model from web images, which is used to initialize and train a video model. Our model applies domain adaptation to account for potential domain shift present between the source domain (webly-supervised data) and target domain and also accounts for noise by adding a novel attention component. We report results competitive with state-of-the-art for webly-supervised approaches on UCF-101 (while simplifying the training process) and also evaluate on Kinetics for comparison.

References in corpus (13)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Deep Residual Learning for Image Recognition
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• The Kinetics Human Action Video Dataset
• Adversarial Discriminative Domain Adaptation
• Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
• A Closer Look at Spatiotemporal Convolutions for Action Recognition
• Learning Spatiotemporal Features with 3D Convolutional Networks
• Non-local Neural Networks
• SlowFast Networks for Video Recognition
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Attend in groups: a weakly-supervised deep learning framework for learning from web data