Towards Universal Representation for Unseen Action Recognition
arXiv:1803.08460
Abstract
Unseen Action Recognition (UAR) aims to recognise novel action categories without training examples. While previous methods focus on inner-dataset seen/unseen splits, this paper proposes a pipeline using a large-scale training source to achieve a Universal Representation (UR) that can generalise to a more realistic Cross-Dataset UAR (CD-UAR) scenario. We first address UAR as a Generalised Multiple-Instance Learning (GMIL) problem and discover 'building-blocks' from the large-scale ActivityNet dataset using distribution kernels. Essential visual and semantic components are preserved in a shared space to achieve the UR that can efficiently generalise to new datasets. Predicted UR exemplars can be improved by a simple semantic adaptation, and then an unseen action can be directly recognised using UR during the test. Without further training, extensive experiments manifest significant improvements over the UCF101 and HMDB51 benchmarks.
arXiv commentAccepted at CVPR 2018
References in corpus (15)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- YouTube-8M: A Large-Scale Video Classification Benchmark
- Long-term Temporal Convolutions for Action Recognition
- Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation
- Hidden Two-Stream Convolutional Networks for Action Recognition
- Objects2action: Classifying and localizing actions without any video example
- Guided Optical Flow Learning
- Learning Attributes Equals Multi-Source Domain Generalization
- Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition
- Semantic Embedding Space for Zero-Shot Action Recognition
- Deep Texture Manifold for Ground Terrain Recognition
- Asynchronous Temporal Fields for Action Recognition
- Alternative Semantic Representations for Zero-Shot Human Action Recognition
Cited by in corpus (6)
- Hidden Two-Stream Convolutional Networks for Action Recognition
- Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
- Multi-Modal Zero-Shot Sign Language Recognition
- Elaborative Rehearsal for Zero-shot Action Recognition
- Random Temporal Skipping for Multirate Video Analysis
- Informed Democracy: Voting-based Novelty Detection for Action Recognition