Few-Shot Transformation of Common Actions into Time and Space
arXiv:2104.02439
Abstract
This paper introduces the task of few-shot common action localization in time and space. Given a few trimmed support videos containing the same but unknown action, we strive for spatio-temporal localization of that action in a long untrimmed query video. We do not require any class labels, interval bounds, or bounding boxes. To address this challenging task, we introduce a novel few-shot transformer architecture with a dedicated encoder-decoder structure optimized for joint commonality learning and localization prediction, without the need for proposals. Experiments on our reorganizations of the AVA and UCF101-24 datasets show the effectiveness of our approach for few-shot common action localization, even when the support videos are noisy. Although we are not specifically designed for common localization in time only, we also compare favorably against the few-shot and one-shot state-of-the-art in this setting. Lastly, we demonstrate that the few-shot transformer is easily extended to common action localization per pixel.
References in corpus (6)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Decoupled Weight Decay Regularization
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- Revisiting Few-shot Activity Detection with Class Similarity Control