Power-Performance Characterization of TinyML Systems

Abstract

TinyML systems are enabling machine learning (ML) inference at the edge. However, there is little quantitative analysis of such systems. This paper presents a systematic performance and power characterization of diverse TinyML applications on microcontrollers (MCUs), spanning neural network models, software libraries, operating systems, and hardware architectures. We focus on the impact of the multiple layers of abstraction that provide higher programmability at the expense of performance and energy efficiency. We propose a model to estimate the costs of different abstraction layers and make recommendations for minimizing those costs. Our findings can help designers with Neural Architecture Search (NAS) and CNN inference optimization on edge devices.

7 pages, 9 figures. Published in Proceedings of the IEEE International Conference on Computer Design (ICCD), 2022

References in corpus (10)

• Hello Edge: Keyword Spotting on Microcontrollers
• Benchmarking TPU, GPU, and CPU Platforms for Deep Learning
• Benchmarking TinyML Systems: Challenges and Direction
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• High-Throughput CNN Inference on Embedded ARM big.LITTLE Multi-Core Processors
• PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors
• Neural Network Inference on Mobile SoCs
• High Performance Zero-Memory Overhead Direct Convolutions
• Measuring what Really Matters: Optimizing Neural Networks for TinyML
• Enabling Large Neural Networks on Tiny Microcontrollers with Swapping