Cross-Class Relevance Learning for Temporal Concept Localization
arXiv:1911.08548
Abstract
We present a novel Cross-Class Relevance Learning approach for the task of temporal concept localization. Most localization architectures rely on feature extraction layers followed by a classification layer which outputs class probabilities for each segment. However, in many real-world applications classes can exhibit complex relationships that are difficult to model with this architecture. In contrast, we propose to incorporate target class and class-related features as input, and learn a pairwise binary model to predict general segment to class relevance. This facilitates learning of shared information between classes, and allows for arbitrary class-specific feature engineering. We apply this approach to the 3rd YouTube-8M Video Understanding Challenge together with other leading models, and achieve first place out of over 280 teams. In this paper we describe our approach and show some empirical results.
References in corpus (7)
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- YouTube-8M: A Large-Scale Video Classification Benchmark
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- Multi-Instance Multi-Label Learning
- Learnable pooling with Context Gating for video classification
- Weakly Supervised Action Localization by Sparse Temporal Pooling Network