PV-NAS: Practical Neural Architecture Search for Video Recognition

Abstract

Recently, deep learning has been utilized to solve video recognition problem due to its prominent representation ability. Deep neural networks for video tasks is highly customized and the design of such networks requires domain experts and costly trial and error tests. Recent advance in network architecture search has boosted the image recognition performance in a large margin. However, automatic designing of video recognition network is less explored. In this study, we propose a practical solution, namely Practical Video Neural Architecture Search (PV-NAS).Our PV-NAS can efficiently search across tremendous large scale of architectures in a novel spatial-temporal network search space using the gradient based search methods. To avoid sticking into sub-optimal solutions, we propose a novel learning rate scheduler to encourage sufficient network diversity of the searched models. Extensive empirical evaluations show that the proposed PV-NAS achieves state-of-the-art performance with much fewer computational resources. 1) Within light-weight models, our PV-NAS-L achieves 78.7% and 62.5% Top-1 accuracy on Kinetics-400 and Something-Something V2, which are better than previous state-of-the-art methods (i.e., TSM) with a large margin (4.6% and 3.4% on each dataset, respectively), and 2) among median-weight models, our PV-NAS-M achieves the best performance (also a new record)in the Something-Something V2 dataset.

References in corpus (14)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Neural Architecture Search with Reinforcement Learning
• The Kinetics Human Action Video Dataset
• AutoAugment: Learning Augmentation Policies from Data
• Efficient Neural Architecture Search via Parameter Sharing
• Large-Scale Evolution of Image Classifiers
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• DetNAS: Backbone Search for Object Detection
• Non-local Neural Networks
• Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
• Reinforced Evolutionary Neural Architecture Search
• AM-LFS: AutoML for Loss Function Search

Cited by in corpus (4)

• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Federated Learning on Non-IID Data: A Survey
• Rethinking Graph Neural Architecture Search from Message-passing
• Searching for TrioNet: Combining Convolution with Local and Global Self-Attention