DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework

Abstract

Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

Published in SenSys2017. Code is available on https://github.com/yscacaca/DeepIoT

References in corpus (11)

• Distilling the Knowledge in a Neural Network
• Recurrent Neural Network Regularization
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Deep Learning with Limited Numerical Precision
• Compressing Deep Convolutional Networks using Vector Quantization
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Convolutional neural networks with low-rank regularization
• MuProp: Unbiased Backpropagation for Stochastic Neural Networks

Cited by in corpus (11)

• Bringing AI To Edge: From Deep Learning's Perspective
• Recurrent Neural Networks: An Embedded Computing Perspective
• Edge Intelligence: Architectures, Challenges, and Applications
• On Lightweight Privacy-Preserving Collaborative Learning for IoT Objects
• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• Joint Design of Measurement Matrix and Sparse Support Recovery Method via Deep Auto-encoder
• Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
• AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications
• RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
• On Lightweight Privacy-Preserving Collaborative Learning for Internet of Things by Independent Random Projections
• Deep Room Recognition Using Inaudible Echos