Optimizing Speech Recognition For The Edge

Abstract

While most deployed speech recognition systems today still run on servers, we are in the midst of a transition towards deployments on edge devices. This leap to the edge is powered by the progression from traditional speech recognition pipelines to end-to-end (E2E) neural architectures, and the parallel development of more efficient neural network topologies and optimization techniques. Thus, we are now able to create highly accurate speech recognizers that are both small and fast enough to execute on typical mobile devices. In this paper, we begin with a baseline RNN-Transducer architecture comprised of Long Short-Term Memory (LSTM) layers. We then experiment with a variety of more computationally efficient layer types, as well as apply optimization techniques like neural connection pruning and parameter quantization to construct a small, high quality, on-device speech recognizer that is an order of magnitude smaller than the baseline system without any optimizations.

References in corpus (13)

• LSTM: A Search Space Odyssey
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Sequence Transduction with Recurrent Neural Networks
• Rethinking the Value of Network Pruning
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks
• Stabilizing the Lottery Ticket Hypothesis
• The unreasonable effectiveness of the forget gate
• Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
• Personalized Speech recognition on mobile devices

Cited by in corpus (17)

• TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference
• PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition
• Version Control of Speaker Recognition Systems
• On the quantization of recurrent neural networks
• ML-EXray: Visibility into ML Deployment on the Edge
• ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition
• Efficient CNN-LSTM based Image Captioning using Neural Network Compression
• Dynamic Sparsity Neural Networks for Automatic Speech Recognition
• Accelerator-Aware Training for Transducer-Based Speech Recognition
• When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute
• Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
• Noisy Training Improves E2E ASR for the Edge
• Analyzing the Quality and Stability of a Streaming End-to-End On-Device Speech Recognizer
• 4-bit Quantization of LSTM-based Speech Recognition Models
• Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
• Dissecting User-Perceived Latency of On-Device E2E Speech Recognition