Reducing Bias in Production Speech Models
arXiv:1705.04400
Abstract
Replacing hand-engineered pipelines with end-to-end deep learning systems has enabled strong results in applications like speech and object recognition. However, the causality and latency constraints of production systems put end-to-end speech models back into the underfitting regime and expose biases in the model that we show cannot be overcome by "scaling up", i.e., training bigger models on more data. In this work we systematically identify and address sources of bias, reducing error rates by up to 20% while remaining practical for deployment. We achieve this by utilizing improved neural architectures for streaming inference, solving optimization issues, and employing strategies that increase audio and label modelling versatility.
References in corpus (9)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Achieving Human Parity in Conversational Speech Recognition
- Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
- Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
- Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
- Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
- Segmental Recurrent Neural Networks for End-to-end Speech Recognition
- Learning Multiscale Features Directly From Waveforms
- Trainable Frontend For Robust and Far-Field Keyword Spotting
Cited by in corpus (6)
- Robust sound event detection in bioacoustic sensor networks
- Exploring Neural Transducers for End-to-End Speech Recognition
- LEAF: A Learnable Frontend for Audio Classification
- Improved training for online end-to-end speech recognition systems
- Learning to detect dysarthria from raw speech
- On Front-end Gain Invariant Modeling for Wake Word Spotting