Pose from Action: Unsupervised Learning of Pose Features based on Motion

Abstract

Human actions are comprised of a sequence of poses. This makes videos of humans a rich and dense source of human poses. We propose an unsupervised method to learn pose features from videos that exploits a signal which is complementary to appearance and can be used as supervision: motion. The key idea is that humans go through poses in a predictable manner while performing actions. Hence, given two poses, it should be possible to model the motion that caused the change between them. We represent each of the poses as a feature in a CNN (Appearance ConvNet) and generate a motion encoding from optical flow maps using a separate CNN (Motion ConvNet). The data for this task is automatically generated allowing us to train without human supervision. We demonstrate the strength of the learned representation by finetuning the trained model for Pose Estimation on the FLIC dataset, for static image action recognition on PASCAL and for action recognition in videos on UCF101 and HMDB51.

References in corpus (9)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Going Deeper with Convolutions
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Unsupervised Learning of Visual Representations using Videos
• Learning to See by Moving
• Flowing ConvNets for Human Pose Estimation in Videos
• Learning image representations tied to ego-motion

Cited by in corpus (7)

• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction
• 3D Self-Supervised Methods for Medical Imaging
• A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
• Unsupervised Representation Learning by Sorting Sequences
• Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
• Exploiting Motion Information from Unlabeled Videos for Static Image Action Recognition