NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks

Abstract

"How much energy is consumed for an inference made by a convolutional neural network (CNN)?" With the increased popularity of CNNs deployed on the wide-spectrum of platforms (from mobile devices to workstations), the answer to this question has drawn significant attention. From lengthening battery life of mobile devices to reducing the energy bill of a datacenter, it is important to understand the energy efficiency of CNNs during serving for making an inference, before actually training the model. In this work, we propose NeuralPower: a layer-wise predictive framework based on sparse polynomial regression, for predicting the serving energy consumption of a CNN deployed on any GPU platform. Given the architecture of a CNN, NeuralPower provides an accurate prediction and breakdown for power and runtime across all layers in the whole network, helping machine learners quickly identify the power, runtime, or energy bottlenecks. We also propose the "energy-precision ratio" (EPR) metric to guide machine learners in selecting an energy-efficient CNN architecture that better trades off the energy consumption and prediction accuracy. The experimental results show that the prediction accuracy of the proposed NeuralPower outperforms the best published model to date, yielding an improvement in accuracy of up to 68.5%. We also assess the accuracy of predictions at the network level, by predicting the runtime, power, and energy of state-of-the-art CNN architectures, achieving an average accuracy of 88.24% in runtime, 88.34% in power, and 97.21% in energy. We comprehensively corroborate the effectiveness of NeuralPower as a powerful framework for machine learners by testing it on different GPU platforms and Deep Learning software tools.

Accepted as a conference paper at ACML 2017

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• One weird trick for parallelizing convolutional neural networks
• Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization

Cited by in corpus (11)

• Quantifying the Carbon Emissions of Machine Learning
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models
• Performance Prediction for Convolutional Neural Networks in Edge Devices
• Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization
• Designing Adaptive Neural Networks for Energy-Constrained Image Classification
• Energy Predictive Models for Convolutional Neural Networks on Mobile Platforms
• Optimising the Performance of Convolutional Neural Networks across Computing Systems using Transfer Learning
• ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection