SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

Abstract

The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment. The Internet of Things (IoT) promises to inject machine learning into many of these every-day objects via tiny, cheap MCUs. However, these resource-impoverished hardware platforms severely limit the complexity of machine learning models that can be deployed. For example, although convolutional neural networks (CNNs) achieve state-of-the-art results on many visual recognition tasks, CNN inference on MCUs is challenging due to severe finite memory limitations. To circumvent the memory challenge associated with CNNs, various alternatives have been proposed that do fit within the memory budget of an MCU, albeit at the cost of prediction accuracy. This paper challenges the idea that CNNs are not suitable for deployment on MCUs. We demonstrate that it is possible to automatically design CNNs which generalize well, while also being small enough to fit onto memory-limited MCUs. Our Sparse Architecture Search method combines neural architecture search with pruning in a single, unified approach, which learns superior models on four popular IoT datasets. The CNNs we find are more accurate and up to $4.35\times$ smaller than previous approaches, while meeting the strict MCU working memory constraint.

References in corpus (14)

• Auto-Encoding Variational Bayes
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• DARTS: Differentiable Architecture Search
• Rethinking the Value of Network Pruning
• Variational Dropout and the Local Reparameterization Trick
• The State of Sparsity in Deep Neural Networks
• Bayesian Compression for Deep Learning
• Variational Dropout Sparsifies Deep Neural Networks
• Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
• A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizations

Cited by in corpus (22)

• MCUNet: Tiny Deep Learning on IoT Devices
• Machine Learning for Microcontroller-Class Hardware: A Review
• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Benchmarking TinyML Systems: Challenges and Direction
• Tiny Machine Learning: Progress and Futures
• Bringing AI To Edge: From Deep Learning's Perspective
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids
• Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices
• MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• Searching for Winograd-aware Quantized Networks
• Software Engineering Approaches for TinyML based IoT Embedded Vision: A Systematic Literature Review
• Can weight sharing outperform random architecture search? An investigation with TuNAS
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• Taxonomy of Saliency Metrics for Channel Pruning
• Sparse Systolic Tensor Array for Efficient CNN Hardware Acceleration
• AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator
• Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices
• VINNAS: Variational Inference-based Neural Network Architecture Search
• ISP4ML: Understanding the Role of Image Signal Processing in Efficient Deep Learning Vision Systems