ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design

Abstract

Currently, the neural network architecture design is mostly guided by the \emph{indirect} metric of computation complexity, i.e., FLOPs. However, the \emph{direct} metric, e.g., speed, also depends on the other factors such as memory access cost and platform characterics. Thus, this work proposes to evaluate the direct metric on the target platform, beyond only considering FLOPs. Based on a series of controlled experiments, this work derives several practical \emph{guidelines} for efficient network design. Accordingly, a new architecture is presented, called \emph{ShuffleNet V2}. Comprehensive ablation experiments verify that our model is the state-of-the-art in terms of speed and accuracy tradeoff.

References in corpus (20)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Squeeze-and-Excitation Networks
• cuDNN: Efficient Primitives for Deep Learning
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Learning Structured Sparsity in Deep Neural Networks
• Channel Pruning for Accelerating Very Deep Neural Networks
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
• Light-Head R-CNN: In Defense of Two-Stage Object Detector
• Progressive Neural Architecture Search
• Distributed Deep Learning Using Synchronous Stochastic Gradient Descent
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• Interleaved Group Convolutions for Deep Neural Networks
• Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network
• IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
• IGCV$2$: Interleaved Structured Sparse Convolutional Neural Networks
• Genetic CNN

Cited by in corpus (47)

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Knowledge Distillation: A Survey
• OCNet: Object Context Network for Scene Parsing
• MixConv: Mixed Depthwise Convolutional Kernels
• SNAS: Stochastic Neural Architecture Search
• Glyce: Glyph-vectors for Chinese Character Representations
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• Training with Quantization Noise for Extreme Model Compression
• Selective Kernel Networks
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
• Measuring the Algorithmic Efficiency of Neural Networks
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks
• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
• K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning
• Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search
• Searching for Low-Bit Weights in Quantized Neural Networks
• FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
• RC-DARTS: Resource Constrained Differentiable Architecture Search
• CARS: Continuous Evolution for Efficient Neural Architecture Search
• Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• $A^2$-Nets: Double Attention Networks
• Trained Rank Pruning for Efficient Deep Neural Networks
• Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
• Efficient Deep Neural Networks
• Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTM
• AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks
• IRLAS: Inverse Reinforcement Learning for Architecture Search
• Seesaw-Net: Convolution Neural Network With Uneven Group Convolution
• A Novel lightweight Convolutional Neural Network, ExquisiteNetV2
• Learning Student Networks via Feature Embedding
• Channel Locality Block: A Variant of Squeeze-and-Excitation
• DeepPeep: Exploiting Design Ramifications to Decipher the Architecture of Compact DNNs
• LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution
• A Simple Method to Reduce Off-chip Memory Accesses on Convolutional Neural Networks
• FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search
• Analysis of Dimensional Influence of Convolutional Neural Networks for Histopathological Cancer Classification
• DiabDeep: Pervasive Diabetes Diagnosis based on Wearable Medical Sensors and Efficient Neural Networks
• Algorithm-hardware Co-design for Deformable Convolution
• Small, Accurate, and Fast Vehicle Re-ID on the Edge: the SAFR Approach
• LIGAR: Lightweight General-purpose Action Recognition
• Parametric Contrastive Learning
• Convolutional neural networks compression with low rank and sparse tensor decompositions
• ESFNet: Efficient Network for Building Extraction from High-Resolution Aerial Images
• Dynamic Multi-path Neural Network
• Natural Disaster Classification using Aerial Photography Explainable for Typhoon Damaged Feature
• Cross-filter compression for CNN inference acceleration