Intelligence Beyond the Edge: Inference on Intermittent Embedded Systems

Abstract

Energy-harvesting technology provides a promising platform for future IoT applications. However, since communication is very expensive in these devices, applications will require inference "beyond the edge" to avoid wasting precious energy on pointless communication. We show that application performance is highly sensitive to inference accuracy. Unfortunately, accurate inference requires large amounts of computation and memory, and energy-harvesting systems are severely resource-constrained. Moreover, energy-harvesting systems operate intermittently, suffering frequent power failures that corrupt results and impede forward progress. This paper overcomes these challenges to present the first full-scale demonstration of DNN inference on an energy-harvesting system. We design and implement SONIC, an intermittence-aware software system with specialized support for DNN inference. SONIC introduces loop continuation, a new technique that dramatically reduces the cost of guaranteeing correct intermittent execution for loop-heavy code like DNN inference. To build a complete system, we further present GENESIS, a tool that automatically compresses networks to optimally balance inference accuracy and energy, and TAILS, which exploits SIMD hardware available in some microcontrollers to improve energy efficiency. Both SONIC & TAILS guarantee correct intermittent execution without any hand-tuning or performance loss across different power systems. Across three neural networks on a commercially available microcontroller, SONIC & TAILS reduce inference energy by 6.9x and 12.2x, respectively, over the state-of-the-art.

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Xception: Deep Learning with Depthwise Separable Convolutions
• Ray: A Distributed Framework for Emerging AI Applications
• LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks
• Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration

Cited by in corpus (5)

• Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI
• Towards Battery-Free Machine Learning and Inference in Underwater Environments
• Regularizing Activation Distribution for Training Binarized Deep Networks
• Intermittent Learning: On-Device Machine Learning on Intermittently Powered System
• Amalgamated Intermittent Computing Systems