Unsupervised Learning of View-invariant Action Representations
arXiv:1809.01844
Abstract
The recent success in human action recognition with deep learning methods mostly adopt the supervised learning paradigm, which requires significant amount of manually labeled data to achieve good performance. However, label collection is an expensive and time-consuming process. In this work, we propose an unsupervised learning framework, which exploits unlabeled data to learn video representations. Different from previous works in video representation learning, our unsupervised learning task is to predict 3D motion in multiple target views using video representation from a source view. By learning to extrapolate cross-view motions, the representation can capture view-invariant motion dynamics which is discriminative for the action. In addition, we propose a view-adversarial training method to enhance learning of view-invariant features. We demonstrate the effectiveness of the learned representations for action recognition on multiple datasets.
arXiv commentNIPS 2018
References in corpus (11)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Striving for Simplicity: The All Convolutional Net
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
- Deep Generative Stochastic Networks Trainable by Backprop
- Video (language) modeling: a baseline for generative models of natural videos
- Unsupervised Visual Representation Learning by Context Prediction
- Spatio-temporal video autoencoder with differentiable memory
- Cross-view Action Modeling, Learning and Recognition
- View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data
- Unsupervised Learning of Long-Term Motion Dynamics for Videos
- Transitive Invariance for Self-supervised Visual Representation Learning
Cited by in corpus (12)
- Synthetic Humans for Action Recognition from Unseen Viewpoints
- Video Storytelling: Textual Summaries for Events
- VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views
- View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose
- 3D Human Action Representation Learning via Cross-View Consistency Pursuit
- Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition
- 3DV: 3D Dynamic Voxel for Action Recognition in Depth Video
- View-Invariant Probabilistic Embedding for Human Pose
- Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning
- View-invariant action recognition
- Unsupervised View-Invariant Human Posture Representation
- PALMAR: Towards Adaptive Multi-inhabitant Activity Recognition in Point-Cloud Technology