Contrastive Video Representation Learning via Adversarial Perturbations
arXiv:1807.09380
Abstract
Adversarial perturbations are noise-like patterns that can subtly change the data, while failing an otherwise accurate classifier. In this paper, we propose to use such perturbations within a novel contrastive learning setup to build negative samples, which are then used to produce improved video representations. To this end, given a well-trained deep model for per-frame video recognition, we first generate adversarial noise adapted to this model. Positive and negative bags are produced using the original data features from the full video sequence and their perturbed counterparts, respectively. Unlike the classic contrastive learning methods, we develop a binary classification problem that learns a set of discriminative hyperplanes -- as a subspace -- that will separate the two bags from each other. This subspace is then used as a descriptor for the video, dubbed \emph{discriminative subspace pooling}. As the perturbed features belong to data classes that are likely to be confused with the original features, the discriminative subspace will characterize parts of the feature space that are more representative of the original data, and thus may provide robust video representations. To learn such descriptors, we formulate a subspace learning objective on the Stiefel manifold and resort to Riemannian optimization methods for solving it efficiently. We provide experiments on several video datasets and demonstrate state-of-the-art results.
Revised version of ECCV 2018 Paper: Learning Discriminative Video Representations Using Adversarial Perturbations
References in corpus (11)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Representation Learning with Contrastive Predictive Coding
- Two-Stream Convolutional Networks for Action Recognition in Videos
- On the difficulty of training Recurrent Neural Networks
- Learning deep representations by mutual information estimation and maximization
- Unsupervised Learning of Video Representations using LSTMs
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Contrastive Multiview Coding
- Spatiotemporal Residual Networks for Video Action Recognition
- Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
- On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization