Domain-Specific Priors and Meta Learning for Few-Shot First-Person Action Recognition

Abstract

The lack of large-scale real datasets with annotations makes transfer learning a necessity for video activity understanding. We aim to develop an effective method for few-shot transfer learning for first-person action classification. We leverage independently trained local visual cues to learn representations that can be transferred from a source domain, which provides primitive action labels, to a different target domain using only a handful of examples. Visual cues we employ include object-object interactions, hand grasps and motion within regions that are a function of hand locations. We employ a framework based on meta-learning to extract the distinctive and domain invariant components of the deployed visual cues. This enables transfer of action classification models across public datasets captured with diverse scene and action configurations. We present comparative results of our transfer learning methodology and report superior results over state-of-the-art action classification approaches for both inter-class and inter-dataset transfer.

arXiv commentPaper has been accepted in Transactions on Pattern Analysis and Machine Intelligence

References in corpus (13)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Prototypical Networks for Few-shot Learning
• Deep Residual Learning for Image Recognition
• On the difficulty of training Recurrent Neural Networks
• A Structured Self-attentive Sentence Embedding
• Action Recognition using Visual Attention
• Object Level Visual Reasoning in Videos
• VideoGraph: Recognizing Minutes-Long Human Activities in Videos
• What have we learned from deep representations for action recognition?
• Towards an Unequivocal Representation of Actions

Cited by in corpus (2)

• Continuous Human Action Recognition for Human-Machine Interaction: A Review
• Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization