MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors

Abstract

In recent years, convolutional networks have demonstrated unprecedented performance in the image restoration task of super-resolution (SR). SR entails the upscaling of a single low-resolution image in order to meet application-specific image quality demands and plays a key role in mobile devices. To comply with privacy regulations and reduce the overhead of cloud computing, executing SR models locally on-device constitutes a key alternative approach. Nevertheless, the excessive compute and memory requirements of SR workloads pose a challenge in mapping SR networks on resource-constrained mobile platforms. This work presents MobiSR, a novel framework for performing efficient super-resolution on-device. Given a target mobile platform, the proposed framework considers popular model compression techniques and traverses the design space to reach the highest performing trade-off between image quality and processing speed. At run time, a novel scheduler dispatches incoming image patches to the appropriate model-engine pair based on the patch's estimated upscaling difficulty in order to meet the required image quality with minimum processing latency. Quantitative evaluation shows that the proposed framework yields on-device SR designs that achieve an average speedup of 2.13x over highly-optimized parallel difficulty-unaware mappings and 4.79x over highly-optimized single compute engine implementations.

Accepted at the 25th Annual International Conference on Mobile Computing and Networking (MobiCom), 2019

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Multi-Scale Dense Networks for Resource Efficient Image Classification
• Shallow-Deep Networks: Understanding and Mitigating Network Overthinking
• Focus: Querying Large Video Datasets with Low Latency and Low Cost
• CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android
• EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices
• NoScope: Optimizing Neural Network Queries over Video at Scale
• Deploying Deep Neural Networks in the Embedded Space

Cited by in corpus (9)

• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
• Artificial Intelligence of Things: A Survey
• SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
• AdaEnlight: Energy-aware Low-light Video Stream Enhancement on Mobile Devices
• NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution
• SoD${}^2$: Statically Optimizing Dynamic Deep Neural Network
• On-Device Text Image Super Resolution
• Neural Enhancement in Content Delivery Systems: The State-of-the-Art and Future Directions