NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications

Abstract

This work proposes an algorithm, called NetAdapt, that automatically adapts a pre-trained deep neural network to a mobile platform given a resource budget. While many existing algorithms simplify networks based on the number of MACs or weights, optimizing those indirect metrics may not necessarily reduce the direct metrics, such as latency and energy consumption. To solve this problem, NetAdapt incorporates direct metrics into its adaptation algorithm. These direct metrics are evaluated using empirical measurements, so that detailed knowledge of the platform and toolchain is not required. NetAdapt automatically and progressively simplifies a pre-trained network until the resource budget is met while maximizing the accuracy. Experiment results show that NetAdapt achieves better accuracy versus latency trade-offs on both mobile CPU and mobile GPU, compared with the state-of-the-art automated network simplification algorithms. For image classification on the ImageNet dataset, NetAdapt achieves up to a 1.7$\times$ speedup in measured inference latency with equal or higher accuracy on MobileNets (V1&V2).

arXiv commentAccepted by ECCV 2018

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Binarized Neural Networks
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures

Cited by in corpus (18)

• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Bringing AI To Edge: From Deep Learning's Perspective
• Edge Intelligence: Architectures, Challenges, and Applications
• One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search
• FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• Hardware-Centric AutoML for Mixed-Precision Quantization
• Fine-Grained Neural Architecture Search
• GeneCAI: Genetic Evolution for Acquiring Compact AI
• CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs
• Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms
• MoGA: Searching Beyond MobileNetV3
• Discovering Multi-Hardware Mobile Models via Architecture Search
• Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public Cloud
• ModiPick: SLA-aware Accuracy Optimization For Mobile Deep Inference
• Compact CNN Structure Learning by Knowledge Distillation
• Single Image Depth Prediction with Wavelet Decomposition