Revisiting Few-shot Activity Detection with Class Similarity Control

Abstract

Many interesting events in the real world are rare making preannotated machine learning ready videos a rarity in consequence. Thus, temporal activity detection models that are able to learn from a few examples are desirable. In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection based on proposal regression which detects the start and end time of the activities in untrimmed videos. Our model is end-to-end trainable, takes into account the frame rate differences between few-shot activities and untrimmed test videos, and can benefit from additional few-shot examples. We experiment on three large scale benchmarks for temporal activity detection (ActivityNet1.2, ActivityNet1.3 and THUMOS14 datasets) in a few-shot setting. We also study the effect on performance of different amount of overlap with activities used to pretrain the video classification backbone and propose corrective measures for future works in this domain. Our code will be made available.

References in corpus (8)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• A Simple Neural Attentive Meta-Learner
• One-shot Learning with Memory-Augmented Neural Networks
• Few-Example Object Detection with Model Communication
• Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks
• Attentive Recurrent Comparators

Cited by in corpus (4)

• Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers
• Few-Shot Action Localization without Knowing Boundaries
• Annotation-Efficient Untrimmed Video Action Recognition
• Few-Shot Transformation of Common Actions into Time and Space