Architecture Aware Latency Constrained Sparse Neural Networks

Abstract

Acceleration of deep neural networks to meet a specific latency constraint is essential for their deployment on mobile devices. In this paper, we design an architecture aware latency constrained sparse (ALCS) framework to prune and accelerate CNN models. Taking modern mobile computation architectures into consideration, we propose Single Instruction Multiple Data (SIMD)-structured pruning, along with a novel sparse convolution algorithm for efficient computation. Besides, we propose to estimate the run time of sparse models with piece-wise linear interpolation. The whole latency constrained pruning task is formulated as a constrained optimization problem that can be efficiently solved with Alternating Direction Method of Multipliers (ADMM). Extensive experiments show that our system-algorithm co-design framework can achieve much better Pareto frontier among network accuracy and latency on resource-constrained mobile devices.

References in corpus (5)

• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
• Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization
• Dynamic Model Pruning with Feedback
• Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM