Temporal Convolution for Real-time Keyword Spotting on Mobile Devices

Abstract

Keyword spotting (KWS) plays a critical role in enabling speech-based user interactions on smart devices. Recent developments in the field of deep learning have led to wide adoption of convolutional neural networks (CNNs) in KWS systems due to their exceptional accuracy and robustness. The main challenge faced by KWS systems is the trade-off between high accuracy and low latency. Unfortunately, there has been little quantitative analysis of the actual latency of KWS models on mobile devices. This is especially concerning since conventional convolution-based KWS approaches are known to require a large number of operations to attain an adequate level of performance. In this paper, we propose a temporal convolution for real-time KWS on mobile devices. Unlike most of the 2D convolution-based KWS approaches that require a deep architecture to fully capture both low- and high-frequency domains, we exploit temporal convolutions with a compact ResNet architecture. In Google Speech Command Dataset, we achieve more than \textbf{385x} speedup on Google Pixel 1 and surpass the accuracy compared to the state-of-the-art model. In addition, we release the implementation of the proposed and the baseline models including an end-to-end pipeline for training models and evaluating them on mobile devices.

In INTERSPEECH 2019

References in corpus (5)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Hello Edge: Keyword Spotting on Microcontrollers
• Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
• A neural attention model for speech command recognition
• Small-footprint Keyword Spotting Using Deep Neural Network and Connectionist Temporal Classifier

Cited by in corpus (17)

• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Streaming keyword spotting on mobile devices
• Neural Architecture Search For Keyword Spotting
• Few-Shot Keyword Spotting With Prototypical Networks
• Multi-task Learning for Voice Trigger Detection
• MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition
• Training Keyword Spotters with Limited and Synthesized Speech Data
• AutoKWS: Keyword Spotting with Differentiable Architecture Search
• Neural ODE with Temporal Convolution and Time Delay Neural Networks for Small-Footprint Keyword Spotting
• EfficientNet-Absolute Zero for Continuous Speech Keyword Spotting
• A Lightweight dynamic filter for keyword spotting
• Encoder-Decoder Neural Architecture Optimization for Keyword Spotting
• FakeWake: Understanding and Mitigating Fake Wake-up Words of Voice Assistants
• A Separable Temporal Convolution Neural Network with Attention for Small-Footprint Keyword Spotting
• SubSpectral Normalization for Neural Audio Data Processing
• AUC Optimization for Robust Small-footprint Keyword Spotting with Limited Training Data
• Separable Temporal Convolution plus Temporally Pooled Attention for Lightweight High-performance Keyword Spotting