Visual Wake Words Dataset

Abstract

The emergence of Internet of Things (IoT) applications requires intelligence on the edge. Microcontrollers provide a low-cost compute platform to deploy intelligent IoT applications using machine learning at scale, but have extremely limited on-chip memory and compute capability. To deploy computer vision on such devices, we need tiny vision models that fit within a few hundred kilobytes of memory footprint in terms of peak usage and model size on device storage. To facilitate the development of microcontroller friendly models, we present a new dataset, Visual Wake Words, that represents a common microcontroller vision use-case of identifying whether a person is present in the image or not, and provides a realistic benchmark for tiny vision models. Within a limited memory footprint of 250 KB, several state-of-the-art mobile models achieve accuracy of 85-90% on the Visual Wake Words dataset. We anticipate the proposed dataset will advance the research on tiny vision models that can push the pareto-optimal boundary in terms of accuracy versus memory usage for microcontroller applications.

10 pages, 4 figures

References in corpus (7)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Trained Ternary Quantization
• Hello Edge: Keyword Spotting on Microcontrollers
• Not All Ops Are Created Equal!
• Rethinking Machine Learning Development and Deployment for Edge Devices

Cited by in corpus (22)

• MCUNet: Tiny Deep Learning on IoT Devices
• Machine Learning for Microcontroller-Class Hardware: A Review
• Benchmarking TinyML Systems: Challenges and Direction
• TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Intelligence at the Extreme Edge: A Survey on Reformable TinyML
• Flower: A Friendly Federated Learning Research Framework
• MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
• RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference
• Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices
• An affordable hardware-aware neural architecture search for deploying convolutional neural networks on ultra-low-power computing platforms
• Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razor
• Searching for Winograd-aware Quantized Networks
• TinyFedTL: Federated Transfer Learning on Tiny Devices
• Neural networks on microcontrollers: saving memory at inference via operator reordering
• Efficient Deep Neural Networks
• Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles
• AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator
• StreamTinyNet: video streaming analysis with spatial-temporal TinyML
• MLPerf Tiny Benchmark
• The Final Frontier: Deep Learning in Space
• On-device Federated Learning with Flower