Self-Paced Video Data Augmentation with Dynamic Images Generated by Generative Adversarial Networks
arXiv:1909.12929
Abstract
There is an urgent need for an effective video classification method by means of a small number of samples. The deficiency of samples could be effectively alleviated by generating samples through Generative Adversarial Networks (GAN), but the generation of videos on a typical category remains to be underexplored since the complex actions and the changeable viewpoints are difficult to simulate. In this paper, we propose a generative data augmentation method for temporal stream of the Temporal Segment Networks with the dynamic image. The dynamic image compresses the motion information of video into a still image, removing the interference factors such as the background. Thus it is easier to generate images with categorical motion information using GAN. We use the generated dynamic images to enhance the features, with regularization achieved as well, thereby to achieve the effect of video augmentation. In order to deal with the uneven quality of generated images, we propose a Self-Paced Selection (SPS) method, which automatically selects the high-quality generated samples to be added to the network training. Our method is verified on two benchmark datasets, HMDB51 and UCF101. The experimental results show that the method can improve the accuracy of video classification under the circumstance of sample insufficiency and sample imbalance.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Spectral Normalization for Generative Adversarial Networks
- Unsupervised Learning of Video Representations using LSTMs
- Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
- Real-time Action Recognition with Enhanced Motion Vector CNNs
- Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs
- Few-Shot Video Classification via Temporal Alignment