Metric-Based Few-Shot Learning for Video Action Recognition

Abstract

In the few-shot scenario, a learner must effectively generalize to unseen classes given a small support set of labeled examples. While a relatively large amount of research has gone into few-shot learning for image classification, little work has been done on few-shot video classification. In this work, we address the task of few-shot video action recognition with a set of two-stream models. We evaluate the performance of a set of convolutional and recurrent neural network video encoder architectures used in conjunction with three popular metric-based few-shot algorithms. We train and evaluate using a few-shot split of the Kinetics 600 dataset. Our experiments confirm the importance of the two-stream setup, and find prototypical networks and pooled long short-term memory network embeddings to give the best performance as few-shot method and video encoder, respectively. For a 5-shot 5-way task, this setup obtains 84.2% accuracy on the test set and 59.4% on a special "challenge" test set, composed of highly confusable classes.

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Prototypical Networks for Few-shot Learning
• A Short Note about Kinetics-600
• Few-Shot Video Classification via Temporal Alignment

Cited by in corpus (6)

• Few-Shot Action Recognition with Compromised Metric via Optimal Transport
• Few-Shot Action Localization without Knowing Boundaries
• A Study of Few-Shot Audio Classification
• One-Shot Open-Set Skeleton-Based Action Recognition
• Informative Sample-Aware Proxy for Deep Metric Learning
• Unifying Few- and Zero-Shot Egocentric Action Recognition