AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Abstract

Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Conventional model compression techniques rely on hand-crafted heuristics and rule-based policies that require domain experts to explore the large design space trading off among model size, speed, and accuracy, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Model Compression (AMC) which leverage reinforcement learning to provide the model compression policy. This learning-based compression policy outperforms conventional rule-based compression policy by having higher compression ratio, better preserving the accuracy and freeing human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than the handcrafted model compression policy for VGG-16 on ImageNet. We applied this automated, push-the-button compression pipeline to MobileNet and achieved 1.81x speedup of measured inference latency on an Android phone and 1.43x speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Continuous control with deep reinforcement learning
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Deep Convolutional Networks using Vector Quantization
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Pruning Filters for Efficient ConvNets
• Learning both Weights and Connections for Efficient Neural Networks
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Fast Training of Convolutional Networks through FFTs

Cited by in corpus (299)

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• AutoML: A Survey of the State-of-the-Art
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• The State of Sparsity in Deep Neural Networks
• SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
• Structured Pruning for Deep Convolutional Neural Networks: A survey
• Point-Voxel CNN for Efficient 3D Deep Learning
• RGB-D Salient Object Detection: A Survey
• GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning
• SpArch: Efficient Architecture for Sparse Matrix Multiplication
• HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• GhostNets on Heterogeneous Devices via Cheap Operations
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• Tiny Machine Learning: Progress and Futures
• Bringing AI To Edge: From Deep Learning's Perspective
• TSM: Temporal Shift Module for Efficient Video Understanding
• NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
• Discrimination-aware Network Pruning for Deep Model Compression
• AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
• New Directions in Automated Traffic Analysis
• Network Pruning via Transformable Architecture Search
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Automated Machine Learning in Practice: State of the Art and Recent Results
• Soft Threshold Weight Reparameterization for Learnable Sparsity
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing
• Linear Mode Connectivity and the Lottery Ticket Hypothesis
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
• DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks
• AutoDS: Towards Human-Centered Automation of Data Science
• Applications and Techniques for Fast Machine Learning in Science
• Dynamic Convolution: Attention over Convolution Kernels
• Layer-adaptive sparsity for the Magnitude-based Pruning
• FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion
• Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision
• Filter Pruning by Switching to Neighboring CNNs with Good Attributes
• Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications
• Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation
• PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
• Neural Network Distiller: A Python Package For DNN Compression Research
• PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• HAQ: Hardware-Aware Automated Quantization with Mixed Precision
• Winning the Lottery with Continuous Sparsification
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• s-LWSR: Super Lightweight Super-Resolution Network
• Pruning Neural Networks at Initialization: Why are We Missing the Mark?
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework
• ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
• CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization
• Bounding Box Regression with Uncertainty for Accurate Object Detection
• PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning
• MobileDets: Searching for Object Detection Architectures for Mobile Accelerators
• Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Survey
• Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
• Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks
• Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
• AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
• Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• An AutoML-based Approach to Multimodal Image Sentiment Analysis
• PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices
• ROMANet: Fine-Grained Reuse-Driven Off-Chip Memory Access Management and Data Organization for Deep Neural Network Accelerators
• ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations
• Tensor Decomposition for Model Reduction in Neural Networks: A Review
• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient
• Learning Sparse Low-Precision Neural Networks With Learnable Regularization
• YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design
• Attention over Parameters for Dialogue Systems
• How Much Automation Does a Data Scientist Want?
• Joint Multi-Dimension Pruning via Numerical Gradient Update
• End-to-End Supermask Pruning: Learning to Prune Image Captioning Models
• Learned Threshold Pruning
• Locally Free Weight Sharing for Network Width Search
• Model Pruning Based on Quantified Similarity of Feature Maps
• AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications
• Conditional Automated Channel Pruning for Deep Neural Networks
• A Closer Look at Structured Pruning for Neural Network Compression
• A Programmable Approach to Neural Network Compression
• Distilled Neural Networks for Efficient Learning to Rank
• AutoQ: Automated Kernel-Wise Neural Network Quantization
• STEERAGE: Synthesis of Neural Networks Using Architecture Search and Grow-and-Prune Methods
• Spectral pruning of fully connected layers: ranking the nodes based on the eigenvalues
• Sparse Transfer Learning via Winning Lottery Tickets
• Knapsack Pruning with Inner Distillation
• Weight-dependent Gates for Network Pruning
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
• Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
• N3H-Core: Neuron-designed Neural Network Accelerator via FPGA-based Heterogeneous Computing Cores
• Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning
• Design Automation for Efficient Deep Learning Computing
• HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search
• Hardware-Centric AutoML for Mixed-Precision Quantization
• FreezeNet: Full Performance by Reduced Storage Costs
• Pruning Self-attentions into Convolutional Layers in Single Path
• Pruning-aware Sparse Regularization for Network Pruning
• Electro-Magnetic Side-Channel Attack Through Learned Denoising and Classification
• DMCP: Differentiable Markov Channel Pruning for Neural Networks
• MoBiNet: A Mobile Binary Network for Image Classification
• Single Shot Structured Pruning Before Training
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
• SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
• Searching for Winograd-aware Quantized Networks
• EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
• Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilation
• Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
• AM-LFS: AutoML for Loss Function Search
• DHP: Differentiable Meta Pruning via HyperNetworks
• Directional Pruning of Deep Neural Networks
• Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• GeneCAI: Genetic Evolution for Acquiring Compact AI
• An Information Theory-inspired Strategy for Automatic Network Pruning
• Neural Network Pruning with Residual-Connections and Limited-Data
• Prediction-Tracking-Segmentation
• Rethinking Class-Discrimination Based CNN Channel Pruning
• Taxonomy of Saliency Metrics for Channel Pruning
• Serving Recurrent Neural Networks Efficiently with a Spatial Accelerator
• TIRAMISU: A Polyhedral Compiler for Dense and Sparse Deep Learning
• Efficient Deep Neural Networks
• MnasFPN: Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices
• Recurrent Spectral Network (RSN): shaping the basin of attraction of a discrete map to reach automated classification
• FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator
• Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
• CUP: Cluster Pruning for Compressing Deep Neural Networks
• DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation
• A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
• A SOT-MRAM-based Processing-In-Memory Engine for Highly Compressed DNN Implementation
• SQuantizer: Simultaneous Learning for Both Sparse and Low-precision Neural Networks
• Training Compact Neural Networks with Binary Weights and Low Precision Activations
• Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Neural 3D Scene Compression via Model Compression
• S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
• Structured Convolutions for Efficient Neural Network Design
• Neural Architecture Search using Property Guided Synthesis
• Anycost GANs for Interactive Image Synthesis and Editing
• SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architectures
• GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
• Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks
• PENNI: Pruned Kernel Sharing for Efficient CNN Inference
• Learning Filter Basis for Convolutional Neural Network Compression
• Learning to Prune Deep Neural Networks via Reinforcement Learning
• Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
• Towards Compact CNNs via Collaborative Compression
• Towards AutoQML: A Cloud-Based Automated Circuit Architecture Search Framework
• Composition of Saliency Metrics for Channel Pruning with a Myopic Oracle
• Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices
• Pruning Deep Neural Networks using Partial Least Squares
• Shapley Value as Principled Metric for Structured Network Pruning
• Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion
• Neural Epitome Search for Architecture-Agnostic Network Compression
• ROMA: Run-Time Object Detection To Maximize Real-Time Accuracy
• An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
• VEGA: Towards an End-to-End Configurable AutoML Pipeline
• Neural Network Compression Via Sparse Optimization
• CDFI: Compression-Driven Network Design for Frame Interpolation
• HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens
• ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning
• Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
• 2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition
• Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-based Approach
• Automated Model Compression by Jointly Applied Pruning and Quantization
• Differentiable Joint Pruning and Quantization for Hardware Efficiency
• Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems
• RARTS: An Efficient First-Order Relaxed Architecture Search Method
• Hessian-Aware Pruning and Optimal Neural Implant
• Grand Challenge of 106-Point Facial Landmark Localization
• FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training
• NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
• Neural Pruning via Growing Regularization
• Searching for Fast Model Families on Datacenter Accelerators
• Data-Independent Structured Pruning of Neural Networks via Coresets
• Network Pruning using Adaptive Exemplar Filters
• Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?
• Computation Resource Allocation Solution in Recommender Systems
• Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
• BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels
• Dynamic Sparse Graph for Efficient Deep Learning
• Distilling with Performance Enhanced Students
• A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
• Saec: Similarity-Aware Embedding Compression in Recommendation Systems
• On the Bounds of Function Approximations
• Image Captioning with Sparse Recurrent Neural Network
• Graph Neural Networks Including Sparse Interpretability
• Training convolutional neural networks with cheap convolutions and online distillation
• Data-Driven Compression of Convolutional Neural Networks
• ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model
• Pruning Filter in Filter
• EPNAS: Efficient Progressive Neural Architecture Search
• State-of-the-art Techniques in Deep Edge Intelligence
• HALP: Hardware-Aware Latency Pruning
• Deep-n-Cheap: An Automated Search Framework for Low Complexity Deep Learning
• Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough
• SASL: Saliency-Adaptive Sparsity Learning for Neural Network Acceleration
• A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs
• ACP: Automatic Channel Pruning via Clustering and Swarm Intelligence Optimization for CNN
• Network Automatic Pruning: Start NAP and Take a Nap
• DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
• AACP: Model Compression by Accurate and Automatic Channel Pruning
• Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
• Automated Deepfake Detection
• Intelligent Trainer for Model-Based Reinforcement Learning
• ESPN: Extremely Sparse Pruned Networks
• Hardware-Aware DNN Compression for Homogeneous Edge Devices
• Follow Your Path: a Progressive Method for Knowledge Distillation
• Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
• BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whitening
• A Feature-map Discriminant Perspective for Pruning Deep Neural Networks
• Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
• UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration
• DECORE: Deep Compression with Reinforcement Learning
• BCNet: Searching for Network Width with Bilaterally Coupled Network
• MUSCO: Multi-Stage Compression of neural networks
• D-SELD: Dataset-Scalable Exemplar LCA-Decoder
• SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning
• Deep Reinforced Attention Learning for Quality-Aware Visual Recognition
• Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices
• Deep Mixture Density Network for Probabilistic Object Detection
• Differentiable Neural Architecture Learning for Efficient Neural Network Design
• AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance
• Towards A Domain-Customized Automated Machine Learning Framework For Networks and Systems
• Hardware-Aware DNN Compression for Homogeneous Edge Devices
• Gaussian Vector: An Efficient Solution for Facial Landmark Detection
• Implicit Filter Sparsification In Convolutional Neural Networks
• Low Precision Constant Parameter CNN on FPGA
• ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search
• Dense Pruning of Pointwise Convolutions in the Frequency Domain
• An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis
• FLAME: A Self-Adaptive Auto-labeling System for Heterogeneous Mobile Processors
• Finding Everything within Random Binary Networks
• Structured Directional Pruning via Perturbation Orthogonal Projection
• ASCAI: Adaptive Sampling for acquiring Compact AI
• Channel Pruning via Multi-Criteria based on Weight Dependency
• Channel Pruning Guided by Classification Loss and Feature Importance
• Deep Convolutional Neural Networks Structured Pruning via Gravity Regularization
• EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM
• Efficient and Robust LiDAR-Based End-to-End Navigation
• Irregularly Tabulated MLP for Fast Point Feature Embedding
• Evolving Transferable Neural Pruning Functions
• AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
• Disentangled Neural Architecture Search
• NeuralScale: Efficient Scaling of Neurons for Resource-Constrained Deep Neural Networks
• Building BROOK: A Multi-modal and Facial Video Database for Human-Vehicle Interaction Research
• Graph Pruning for Model Compression
• Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks
• Gator: Customizable Channel Pruning of Neural Networks with Gating
• Practitioner Motives to Use Different Hyperparameter Optimization Methods
• Mobiprox: Supporting Dynamic Approximate Computing on Mobiles
• A Highly Effective Low-Rank Compression of Deep Neural Networks with Modified Beam-Search and Modified Stable Rank
• Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search
• Incremental Training and Group Convolution Pruning for Runtime DNN Performance Scaling on Heterogeneous Embedded Platforms
• Optimising Resource Management for Embedded Machine Learning
• Pruning-aware Loss Functions for STOI-Optimized Pruned Recurrent Autoencoders for the Compression of the Stimulation Patterns of Cochlear Implants at Zero Delay
• Tricks and Plug-ins for Gradient Boosting in Image Classification
• Width Transfer: On the (In)variance of Width Optimization
• Carrying out CNN Channel Pruning in a White Box
• HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks
• Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translation
• Effective Model Compression via Stage-wise Pruning
• Attn-HybridNet: Improving Discriminability of Hybrid Features with Attention Fusion
• Coarse and fine-grained automatic cropping deep convolutional neural network
• iWash: A Smartwatch Handwashing Quality Assessment and Reminder System with Real-time Feedback in the Context of Infectious Disease
• Grow-Push-Prune: aligning deep discriminants for effective structural network compression
• Dep-$L_0$: Improving $L_0$-based Network Sparsification via Dependency Modeling
• GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
• AIP: Adversarial Iterative Pruning Based on Knowledge Transfer for Convolutional Neural Networks
• Communication-Computation Efficient Device-Edge Co-Inference via AutoML
• Architecture Aware Latency Constrained Sparse Neural Networks
• AutoSmart: An Efficient and Automatic Machine Learning framework for Temporal Relational Data
• Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
• Multi-Scale Aligned Distillation for Low-Resolution Detection
• SMOF: Squeezing More Out of Filters Yields Hardware-Friendly CNN Pruning
• OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
• A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
• Neural Architecture Search with an Efficient Multiobjective Evolutionary Framework
• Bayesian Sparsification Methods for Deep Complex-valued Networks
• Third ArchEdge Workshop: Exploring the Design Space of Efficient Deep Neural Networks
• Deep Model Compression Via Two-Stage Deep Reinforcement Learning
• Out-of-the-box channel pruned networks
• A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
• MicroNet for Efficient Language Modeling
• Feature Statistics Guided Efficient Filter Pruning
• Conditional Neural Architecture Search
• ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection
• Channel selection using Gumbel Softmax
• An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition
• MICIK: MIning Cross-Layer Inherent Similarity Knowledge for Deep Model Compression
• A Survey on Green Deep Learning
• Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approach