AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
arXiv:2007.15796
Abstract
Action recognition is an open and challenging problem in computer vision. While current state-of-the-art models offer excellent recognition results, their computational expense limits their impact for many real-world applications. In this paper, we propose a novel approach, called AR-Net (Adaptive Resolution Network), that selects on-the-fly the optimal resolution for each frame conditioned on the input for efficient action recognition in long untrimmed videos. Specifically, given a video frame, a policy network is used to decide what input resolution should be used for processing by the action recognition model, with the goal of improving both accuracy and efficiency. We efficiently train the policy network jointly with the recognition model using standard back-propagation. Extensive experiments on several challenging action recognition benchmark datasets well demonstrate the efficacy of our proposed approach over state-of-the-art methods. The project page can be found at https://mengyuest.github.io/AR-Net
References in corpus (12)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Categorical Reparameterization with Gumbel-Softmax
- The Kinetics Human Action Video Dataset
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Compressing Neural Networks with the Hashing Trick
- Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
- Adaptive Computation Time for Recurrent Neural Networks
- Conditional Computation in Neural Networks for faster models
- More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregation
- Deciding How to Decide: Dynamic Routing in Artificial Neural Networks
- LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition