Learning Class Regularized Features for Action Recognition
arXiv:2002.02651 · doi:10.3390/app10186241
Abstract
Training Deep Convolutional Neural Networks (CNNs) is based on the notion of using multiple kernels and non-linearities in their subsequent activations to extract useful features. The kernels are used as general feature extractors without specific correspondence to the target class. As a result, the extracted features do not correspond to specific classes. Subtle differences between similar classes are modeled in the same way as large differences between dissimilar classes. To overcome the class-agnostic use of kernels in CNNs, we introduce a novel method named Class Regularization that performs class-based regularization of layer activations. We demonstrate that this not only improves feature search during training, but also allows an explicit assignment of features per class during each stage of the feature extraction process. We show that using Class Regularization blocks in state-of-the-art CNN architectures for action recognition leads to systematic improvement gains of 1.8%, 1.2% and 1.4% on the Kinetics, UCF-101 and HMDB-51 datasets, respectively.
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Two-Stream Convolutional Networks for Action Recognition in Videos
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- Spatiotemporal Residual Networks for Video Action Recognition
- Video Classification with Channel-Separated Convolutional Networks
- Analyzing Human-Human Interactions: A Survey
- Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutions
- Learn to cycle: Time-consistent feature discovery for action recognition
- Spatio-Temporal FAST 3D Convolutions for Human Action Recognition
- On the Units of GANs (Extended Abstract)
- Class Feature Pyramids for Video Explanation