AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
arXiv:2102.05775
Abstract
Temporal modelling is the key for efficient video action recognition. While understanding temporal information can improve recognition accuracy for dynamic actions, removing temporal redundancy and reusing past features can significantly save computation leading to efficient action recognition. In this paper, we introduce an adaptive temporal fusion network, called AdaFuse, that dynamically fuses channels from current and past feature maps for strong temporal modelling. Specifically, the necessary information from the historical convolution feature maps is fused with current pruned feature maps with the goal of improving both recognition accuracy and efficiency. In addition, we use a skipping operation to further reduce the computation cost of action recognition. Extensive experiments on Something V1 & V2, Jester and Mini-Kinetics show that our approach can achieve about 40% computation savings with comparable accuracy to state-of-the-art methods. The project page can be found at https://mengyuest.github.io/AdaFuse/
Accepted to ICLR2021
References in corpus (19)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Categorical Reparameterization with Gumbel-Softmax
- The Kinetics Human Action Video Dataset
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Adaptive Computation Time for Recurrent Neural Networks
- CondConv: Conditionally Parameterized Convolutions for Efficient Inference
- Dynamic Channel Pruning: Feature Boosting and Suppression
- Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference
- SlowFast Networks for Video Recognition
- Conditional Computation in Neural Networks for faster models
- Channel Gating Neural Networks
- GhostNet: More Features from Cheap Operations
- More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregation
- Dynamic Convolution: Attention over Convolution Kernels
- Deciding How to Decide: Dynamic Routing in Artificial Neural Networks
- On the effectiveness of task granularity for transfer learning
- LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition
- Dynamic Inference: A New Approach Toward Efficient Video Action Recognition