41 citations · 149 across the 46 of their papers we have counts for
24 papers · 1 filter
Conversational Speech Naturalness Predictor
Anfeng Xu, Yashesh Gaur, Naoyuki Kanda +6
Evaluation of conversational naturalness is essential for developing human-like speech agents. However, existing speech naturalness predictors are often designed to assess utteranc…
Transcribing and Translating, Fast and Slow: Joint Speech Translation and Recognition
Niko Moritz, Ruiming Xie, Yashesh Gaur +5
We propose the joint speech translation and recognition (JSTAR) model that leverages the fast-slow cascaded encoder architecture for simultaneous end-to-end automatic speech recogn…
Textless Streaming Speech-to-Speech Translation using Semantic Speech Tokens
Jinzheng Zhao, Niko Moritz, Egor Lakomkin +7
Cascaded speech-to-speech translation systems often suffer from the error accumulation problem and high latency, which is a result of cascaded modules whose inference delays accumu…
Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech
Wonjune Kang, Junteng Jia, Chunyang Wu +8
This work studies the capabilities of a large language model (LLM) to understand paralinguistic aspects of speech without fine-tuning its weights. We utilize an end-to-end system w…
On decoder-only architecture for speech-to-text and large language model integration
Jian Wu, Yashesh Gaur, Zhuo Chen +8
Large language models (LLMs) have achieved remarkable success in the field of natural language processing, enabling better human-computer interaction using natural language. Howeve…
Acoustic-aware Non-autoregressive Spell Correction with Mask Sample Decoding
Ruchao Fan, Guoli Ye, Yashesh Gaur +1
Masked language model (MLM) has been widely used for understanding tasks, e.g. BERT. Recently, MLM has also been used for generation tasks. The most popular one in speech is using…
CTCBERT: Advancing Hidden-unit BERT with CTC Objectives
Ruchao Fan, Yiming Wang, Yashesh Gaur +1
In this work, we present a simple but effective method, CTCBERT, for advancing hidden-unit BERT (HuBERT). HuBERT applies a frame-level cross-entropy (CE) loss, which is similar to…
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu +7
This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize ``who spoke what'' with low latency even when multiple people are…
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu +7
This paper proposes a token-level serialized output training (t-SOT), a novel framework for streaming multi-talker automatic speech recognition (ASR). Unlike existing streaming mul…
Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR
Naoyuki Kanda, Xiong Xiao, Yashesh Gaur +4
This paper presents Transcribe-to-Diarize, a new approach for neural speaker diarization that uses an end-to-end (E2E) speaker-attributed automatic speech recognition (SA-ASR). The…
Continuous Streaming Multi-Talker ASR with Dual-path Transducers
Desh Raj, Liang Lu, Zhuo Chen +2
Streaming recognition of multi-talker conversations has so far been evaluated only for 2-speaker single-turn sessions. In this paper, we investigate it for multi-turn meetings cont…
A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio
Naoyuki Kanda, Xiong Xiao, Jian Wu +6
Speaker-attributed automatic speech recognition (SA-ASR) is a task to recognize "who spoke what" from multi-talker recordings. An SA-ASR system usually consists of multiple modules…
Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone
Naoyuki Kanda, Guoli Ye, Yu Wu +5
Transcribing meetings containing overlapped speech with only a single distant microphone (SDM) has been one of the most challenging problems for automatic speech recognition (ASR).…
End-to-End Speaker-Attributed ASR with Transformer
Naoyuki Kanda, Guoli Ye, Yashesh Gaur +4
This paper presents our recent effort on end-to-end speaker-attributed automatic speech recognition, which jointly performs speaker counting, speech recognition and speaker identif…
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Naoyuki Kanda, Yashesh Gaur +6
The efficacy of external language model (LM) integration with existing end-to-end (E2E) automatic speech recognition (ASR) systems can be improved significantly using the internal…
Exploring End-to-End Multi-channel ASR with Bias Information for Meeting Transcription
Xiaofei Wang, Naoyuki Kanda, Yashesh Gaur +3
Joint optimization of multi-channel front-end and automatic speech recognition (ASR) has attracted much interest. While promising results have been reported for various tasks, past…
Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations
Shahram Ghorbani, Yashesh Gaur, Yu Shi +1
In this study, we try to address the problem of leveraging visual signals to improve Automatic Speech Recognition (ASR), also known as visual context-aware ASR (VC-ASR). We explore…
Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR
Naoyuki Kanda, Zhong Meng, Liang Lu +4
Recently, an end-to-end speaker-attributed automatic speech recognition (E2E SA-ASR) model was proposed as a joint model of speaker counting, speech recognition and speaker identif…
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
Zhong Meng, Sarangarajan Parthasarathy, Eric Sun +7
The external language models (LM) integration remains a challenging task for end-to-end (E2E) automatic speech recognition (ASR) which has no clear division between acoustic and la…
Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings
Naoyuki Kanda, Xuankai Chang, Yashesh Gaur +4
Recently, an end-to-end (E2E) speaker-attributed automatic speech recognition (SA-ASR) model was proposed as a joint model of speaker counting, speech recognition and speaker ident…
Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers
Naoyuki Kanda, Yashesh Gaur, Xiaofei Wang +4
We propose an end-to-end speaker-attributed automatic speech recognition model that unifies speaker counting, speech recognition, and speaker identification on monaural overlapped…
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Jinyu Li, Yu Wu, Yashesh Gaur +3
Recently, there has been a strong push to transition from hybrid models to end-to-end (E2E) models for automatic speech recognition. Currently, there are three promising E2E method…
Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
Zhong Meng, Jinyu Li, Yashesh Gaur +1
Teacher-student (T/S) has shown to be effective for domain adaptation of deep neural network acoustic models in hybrid speech recognition systems. In this work, we extend the T/S l…
Character-Aware Attention-Based End-to-End Speech Recognition
Zhong Meng, Yashesh Gaur, Jinyu Li +1
Predicting words and subword units (WSUs) as the output has shown to be effective for the attention-based encoder-decoder (AED) model in end-to-end speech recognition. However, as…