29 citations · 123 across the 19 of their papers we have counts for
19 papers
Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding
Suyoun Kim, Akshat Shrivastava, Duc Le +3
End-to-end (E2E) spoken language understanding (SLU) systems that generate a semantic parse from speech have become more promising recently. This approach uses a single model that…
Introducing Semantics into Speech Encoders
Derek Xu, Shuyan Dong, Changhan Wang +10
Recent studies find existing self-supervised speech encoders contain primarily acoustic rather than semantic information. As a result, pipelined supervised automatic speech recogni…
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition
Suyoun Kim, Ke Li, Lucas Kabela +4
Recently, there has been an increasing interest in two-pass streaming end-to-end speech recognition (ASR) that incorporates a 2nd-pass rescoring model on top of the conventional 1s…
Deliberation Model for On-Device Spoken Language Understanding
Duc Le, Akshat Shrivastava, Paden Tomasello +4
We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU), where a streaming automatic speech recognition (ASR) model produces the firs…
Evaluating User Perception of Speech Recognition System Quality with Semantic Distance Metric
Suyoun Kim, Duc Le, Weiyi Zheng +6
Measuring automatic speech recognition (ASR) system quality is critical for creating user-satisfying voice-driven applications. Word Error Rate (WER) has been traditionally used to…
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding
Suyoun Kim, Abhinav Arora, Duc Le +4
Word Error Rate (WER) has been the predominant metric used to evaluate the performance of automatic speech recognition (ASR) systems. However, WER is sometimes not a good indicator…
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion
Duc Le, Mahaveer Jain, Gil Keren +9
How to leverage dynamic contextual information in end-to-end speech recognition has remained an active research area. Previous solutions to this problem were either designed for sp…
Improving RNN Transducer Based ASR with Auxiliary Tasks
Chunxi Liu, Frank Zhang, Duc Le +3
End-to-end automatic speech recognition (ASR) models with a single neural network have recently demonstrated state-of-the-art results compared to conventional hybrid speech recogni…
Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer
Suyoun Kim, Yuan Shangguan, Jay Mahadeokar +4
Recurrent Neural Network Transducer (RNN-T), like most end-to-end speech recognition model architectures, has an implicit neural network language model (NNLM) and cannot easily lev…
Cross-Attention End-to-End ASR for Two-Party Conversations
Suyoun Kim, Siddharth Dalmia, Florian Metze
We present an end-to-end speech recognition model that learns interaction between two speakers based on the turn-changing information. Unlike conventional speech recognition models…
Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion
Suyoun Kim, Siddharth Dalmia, Florian Metze
We present a novel conversational-context aware end-to-end speech recognizer based on a gated neural network that incorporates conversational-context/word/speech embeddings. Unlike…
Acoustic-to-Word Models with Conversational Context Information
Suyoun Kim, Florian Metze
Conversational context information, higher-level knowledge that spans across sentences, can help to recognize a long conversation. However, existing speech recognition models are t…
Dialog-context aware end-to-end speech recognition
Suyoun Kim, Florian Metze
Existing speech recognition systems are typically built at the sentence level, although it is known that dialog context, e.g. higher-level knowledge that spans across sentences or…
Towards Language-Universal End-to-End Speech Recognition
Suyoun Kim, Michael L. Seltzer
Building speech recognizers in multiple languages typically involves replicating a monolingual training recipe for each language, or utilizing a multi-task learning approach where…
Improved training for online end-to-end speech recognition systems
Suyoun Kim, Michael L. Seltzer, Jinyu Li +1
Achieving high accuracy with end-to-end speech recognizers requires careful parameter initialization prior to training. Otherwise, the networks may fail to find a good local optimu…
Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning
Suyoun Kim, Takaaki Hori, Shinji Watanabe
Recently, there has been an increasing interest in end-to-end speech recognition that directly transcribes speech to text without any predefined alignments. One approach is the att…
Environmental Noise Embeddings for Robust Speech Recognition
Suyoun Kim, Bhiksha Raj, Ian Lane
We propose a novel deep neural network architecture for speech recognition that explicitly employs knowledge of the background environmental noise within a deep neural network acou…
Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition
Suyoun Kim, Ian Lane
Integration of multiple microphone data is one of the key ways to achieve robust speech recognition in noisy environments or when the speaker is located at some distance from the i…
Multimodal Transfer Deep Learning with Applications in Audio-Visual Recognition
Seungwhan Moon, Suyoun Kim, Haohan Wang
We propose a transfer deep learning (TDL) framework that can transfer the knowledge obtained from a single-modal neural network to a network with a different modality. Specifically…