The Microsoft 2017 Conversational Speech Recognition System

Abstract

We describe the 2017 version of Microsoft's conversational speech recognition system, in which we update our 2016 system with recent developments in neural-network-based acoustic and language modeling to further advance the state of the art on the Switchboard speech recognition task. The system adds a CNN-BLSTM acoustic model to the set of model architectures we combined previously, and includes character-based and dialog session aware LSTM language models in rescoring. For system combination we adopt a two-stage approach, whereby subsets of acoustic models are first combined at the senone/frame level, followed by a word-level voting via confusion networks. We also added a confusion network rescoring step after system combination. The resulting system yields a 5.1\% word error rate on the 2000 Switchboard evaluation set.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Achieving Human Parity in Conversational Speech Recognition
• The Microsoft 2016 Conversational Speech Recognition System
• Comparing Human and Machine Errors in Conversational Speech Transcription

Cited by in corpus (83)

• Long short-term memory networks in memristor crossbars
• High-Throughput In-Memory Computing for Binary Deep Neural Networks with Monolithically Integrated RRAM and 90nm CMOS
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• NSML: Meet the MLaaS platform with a real-world case study
• Forecasting significant stock price changes using neural networks
• Advances in All-Neural Speech Recognition
• Recent Progress in the CUHK Dysarthric Speech Recognition System
• The CAPIO 2017 Conversational Speech Recognition System
• Dive into Deep Learning
• Convolutional neural network for breathing phase detection in lung sounds
• Machine learning for music genre: multifaceted review and experimentation with audioset
• Depth with Nonlinearity Creates No Bad Local Minima in ResNets
• Investigation of Data Augmentation Techniques for Disordered Speech Recognition
• Measuring the Accuracy of Automatic Speech Recognition Solutions
• Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• End-to-end training of time domain audio separation and recognition
• Direct Speech-to-image Translation
• Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition
• Impact of Artificial Intelligence on Businesses: from Research, Innovation, Market Deployment to Future Shifts in Business Models
• Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech
• A High-Fidelity Open Embodied Avatar with Lip Syncing and Expression Capabilities
• End-to-End Far-Field Speech Recognition with Unified Dereverberation and Beamforming
• Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction
• Feature Engineering and Forecasting via Derivative-free Optimization and Ensemble of Sequence-to-sequence Networks with Applications in Renewable Energy
• Platform for Situated Intelligence
• Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data
• LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring
• Improving Readability for Automatic Speech Recognition Transcription
• SAPIEN: Affective Virtual Agents Powered by Large Language Models
• A Comparison of Online Automatic Speech Recognition Systems and the Nonverbal Responses to Unintelligible Speech
• Deep learning for photoacoustic imaging: a survey
• Meeting Transcription Using Virtual Microphone Arrays
• Fast and Robust Unsupervised Contextual Biasing for Speech Recognition
• English Broadcast News Speech Recognition by Humans and Machines
• HarperValleyBank: A Domain-Specific Spoken Dialog Corpus
• Adapting Long Context NLM for ASR Rescoring in Conversational Agents
• A Benchmarking on Cloud based Speech-To-Text Services for French Speech and Background Noise Effect
• Efficient Neural Network Approaches for Leather Defect Classification
• An Empirical Study of Efficient ASR Rescoring with Transformers
• Speech Pattern based Black-box Model Watermarking for Automatic Speech Recognition
• Improving noise robust automatic speech recognition with single-channel time-domain enhancement network
• Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework
• Comparison of Lattice-Free and Lattice-Based Sequence Discriminative Training Criteria for LVCSR
• Evaluating Models of Robust Word Recognition with Serial Reproduction
• Large-Scale News Classification using BERT Language Model: Spark NLP Approach
• A Bayesian Approach to Recurrence in Neural Networks
• Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model
• A Cycle-GAN Approach to Model Natural Perturbations in Speech for ASR Applications
• Particle Filter Recurrent Neural Networks
• M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention
• Improving N-gram Language Models with Pre-trained Deep Transformer
• Multi-view Frequency LSTM: An Efficient Frontend for Automatic Speech Recognition
• ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition
• Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Deep Learning Application in Security and Privacy -- Theory and Practice: A Position Paper
• Deep Learning for Automated Medical Image Analysis
• On the Application of Data-Driven Deep Neural Networks in Linear and Nonlinear Structural Dynamics
• Practical Speech Re-use Prevention in Voice-driven Services
• Communication Access Real-Time Translation Through Collaborative Correction of Automatic Speech Recognition
• SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognition
• Utterance-level Permutation Invariant Training with Latency-controlled BLSTM for Single-channel Multi-talker Speech Separation
• Deep segmental phonetic posterior-grams based discovery of non-categories in L2 English speech
• Broken Access: On the Challenges of Screen Reader Assisted Two-Factor and Passwordless Authentication
• ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees
• Look Who's Talking: Active Speaker Detection in the Wild
• The Marchex 2018 English Conversational Telephone Speech Recognition System
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis
• Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
• Advancing automatic speech recognition using feature fusion with self-supervised learning features: A case study on Fearless Steps Apollo corpus
• Searching Personal Collections
• Drivers' Manoeuvre Modelling and Prediction for Safe HRI
• Edge service resource allocation strategy based on intelligent prediction
• Towards Better Understanding of Spontaneous Conversations: Overcoming Automatic Speech Recognition Errors With Intent Recognition
• Graphs as Tools to Improve Deep Learning Methods
• Leveraging Cross-Utterance Context For ASR Decoding
• Context-aware RNNLM Rescoring for Conversational Speech Recognition
• Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model
• User Intent Inference for Web Search and Conversational Agents
• The NTNU System at the Interspeech 2020 Non-Native Children's Speech ASR Challenge
• Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions