Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions

Abstract

Visual recognition is currently one of the most important and active research areas in computer vision, pattern recognition, and even the general field of artificial intelligence. It has great fundamental importance and strong industrial needs. Deep neural networks (DNNs) have largely boosted their performances on many concrete tasks, with the help of large amounts of training data and new powerful computation resources. Though recognition accuracy is usually the first concern for new progresses, efficiency is actually rather important and sometimes critical for both academic research and industrial applications. Moreover, insightful views on the opportunities and challenges of efficiency are also highly required for the entire community. While general surveys on the efficiency issue of DNNs have been done from various perspectives, as far as we are aware, scarcely any of them focused on visual recognition systematically, and thus it is unclear which progresses are applicable to it and what else should be concerned. In this paper, we present the review of the recent advances with our suggestions on the new possible directions towards improving the efficiency of DNN-related visual recognition approaches. We investigate not only from the model but also the data point of view (which is not the case in existing surveys), and focus on three most studied data types (images, videos and points). This paper attempts to provide a systematic summary via a comprehensive survey which can serve as a valuable reference and inspire both researchers and practitioners who work on visual recognition problems.

References in corpus (46)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Neural Architecture Search with Reinforcement Learning
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• cuDNN: Efficient Primitives for Deep Learning
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Trained Ternary Quantization
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Compressing Neural Networks with the Hashing Trick
• Designing Neural Network Architectures using Reinforcement Learning
• Fast-SCNN: Fast Semantic Segmentation Network
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Quasi-Recurrent Neural Networks
• Efficient Architecture Search by Network Transformation
• Slimmable Neural Networks
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• WRPN: Wide Reduced-Precision Networks
• Comparing Rewinding and Fine-tuning in Neural Network Pruning
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding
• Compressing Recurrent Neural Network with Tensor Train
• Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition
• Ultimate tensorization: compressing convolutional and FC layers alike
• FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
• Training Quantized Nets: A Deeper Understanding
• Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
• Proving the Lottery Ticket Hypothesis: Pruning is All You Need
• Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds
• Tensor-Train Recurrent Neural Networks for Video Classification
• VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization
• Hybrid Tensor Decomposition in Neural Network Compression
• Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
• Dual Path Networks
• Sharing Residual Units Through Collective Tensor Factorization in Deep Neural Networks
• Compressing Recurrent Neural Networks Using Hierarchical Tucker Tensor Decomposition
• Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Need
• Adaptive Quantization for Deep Neural Network
• Towards Understanding the Transferability of Deep Representations
• Tensor Contraction Layers for Parsimonious Deep Nets
• Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm
• Singular-Value-Decomposition Analysis of Associative Memory in a Neural Network
• URNet : User-Resizable Residual Networks with Conditional Gating Module
• Recurrent Convolution for Compact and Cost-Adjustable Neural Networks: An Empirical Study
• CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compression