EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices

Abstract

In recent years, advances in deep learning have resulted in unprecedented leaps in diverse tasks spanning from speech and object recognition to context awareness and health monitoring. As a result, an increasing number of AI-enabled applications are being developed targeting ubiquitous and mobile devices. While deep neural networks (DNNs) are getting bigger and more complex, they also impose a heavy computational and energy burden on the host devices, which has led to the integration of various specialized processors in commodity devices. Given the broad range of competing DNN architectures and the heterogeneity of the target hardware, there is an emerging need to understand the compatibility between DNN-platform pairs and the expected performance benefits on each platform. This work attempts to demystify this landscape by systematically evaluating a collection of state-of-the-art DNNs on a wide variety of commodity devices. In this respect, we identify potential bottlenecks in each architecture and provide important guidelines that can assist the community in the co-design of more efficient DNNs and accelerators.

Accepted at MobiSys 2019: 3rd International Workshop on Embedded and Mobile Deep Learning (EMDL), 2019

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning
• pCAMP: Performance Comparison of Machine Learning Packages on the Edges

Cited by in corpus (5)

• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
• It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation
• Smart at what cost? Characterising Mobile Deep Neural Networks in the wild
• Performance landscape of resource-constrained platforms targeting DNNs