5 citations · 15 across the 22 of their papers we have counts for
23 papers
GigaSpeechBench: A Real-World Multilingual Speech-to-Text Benchmark
Yujie Tu, Yifan Yang, Tianrui Wang +35
While modern ASR systems achieve low error rates on high-resource benchmarks, such performance often overestimates real-world robustness. Existing evaluations address challenges in…
Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech
Yue Heng Yeo, Haoyang Li, Yizhou Peng +6
Code-switch (CS) Automatic Speech Recognition (ASR) remains challenging due to limited availability of high quality CS text-speech pairs for training. Although synthetic data augme…
DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action
Haoyang Zhang, Jun Chen, Donghang Wu +13
Recent advances in spoken dialogue language models have shifted from turn-based to full-duplex designs, where the model continuously listens to the user while generating responses.…
Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems
Yizhou Peng, Ziyang Ma, Changsong Liu +3
Cascaded Automatic Speech Recognition - Large Language Model (ASR-LLM) pipelines remain popular for industrial Spoken Dialogue Systems (SDS), primarily because their decoupled desi…
Evaluating the Expressive Appropriateness of Speech in Rich Contexts
Tianrui Wang, Ziyang Ma, Yizhou Peng +26
Evaluating expressive speech remains challenging, as existing methods mainly assess emotional intensity and overlook whether a speech sample is expressively appropriate for its con…
Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input
Changsong Liu, Tianrui Wang, Ye Ni +2
Streaming TTS that receives streaming text is essential for interactive systems, yet this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-for…
Improving Code-Switching Speech Recognition with TTS Data Augmentation
Yue Heng Yeo, Yuchen Hu, Shreyas Gopal +3
Automatic speech recognition (ASR) for conversational code-switching speech remains challenging due to the scarcity of realistic, high-quality labeled speech data. This paper explo…
POTSA: A Cross-Lingual Speech Alignment Framework for Speech-to-Text Translation
Xuanchen Li, Chenrui Cui, Tianrui Wang +9
Speech Large Language Models have achieved breakthroughs in multilingual speech-to-text translation. However, existing approaches often overlook semantic commonalities across sourc…
Cross-modal Consistency Guidance for Robust Emotion Control in Auto-Regressive TTS Models
Yizhou Peng, Yukun Ma, Chong Zhang +4
While Text-to-Speech (TTS) systems enable emotional control via natural-language instructions, expressiveness, naturalness, and speech quality degrade when the target emotion confl…
Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendations
Yihao Wu, Tianrui Wang, Yizhou Peng +5
While biases in large language models (LLMs), such as stereotypes and cultural tendencies in outputs, have been examined and identified, their presence and characteristics in spoke…
Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciation
Changsong Liu, Yizhou Peng, Eng Siong Chng
Contextual automatic speech recognition (ASR) systems allow for recognizing out-of-vocabulary (OOV) words, such as named entities or rare words. However, it remains challenging due…
FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systems
Yizhou Peng, Yi-Wen Chao, Dianwen Ng +4
Full-duplex spoken dialogue systems (FDSDS) enable more natural human-machine interactions by allowing real-time user interruptions and backchanneling, compared to traditional SDS…
Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR
Yizhou Peng, Hexin Liu, Eng Siong Chng
This paper introduces the integration of language-specific bi-directional context into a speech large language model (SLLM) to improve multilingual continuous conversational automa…
NTU Speechlab LLM-Based Multilingual ASR System for Interspeech MLC-SLM Challenge 2025
Yizhou Peng, Bin Wang, Yi-Wen Chao +5
This report details the NTU Speechlab system developed for the Interspeech 2025 Multilingual Conversational Speech and Language Model (MLC-SLM) Challenge (Task I), where we achieve…
Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR
Hongli Yang, Sheng Li, Hao Huang +2
Recent advancements in multilingual automatic speech recognition (ASR) have been driven by large-scale end-to-end models like Whisper. However, challenges such as language interfer…
Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning
Hongli Yang, Yizhou Peng, Hao Huang +1
Large-scale multilingual ASR models like Whisper excel in high-resource settings but face challenges in low-resource scenarios, such as rare languages and code-switching (CS), due…
Bridging Speech and Text: Enhancing ASR with Pinyin-to-Character Pre-training in LLMs
Yang Yuhang, Peng Yizhou, Eng Siong Chng +1
The integration of large language models (LLMs) with pre-trained speech models has opened up new avenues in automatic speech recognition (ASR). While LLMs excel in multimodal under…
Adapting OpenAI's Whisper for Speech Recognition on Code-Switch Mandarin-English SEAME and ASRU2019 Datasets
Yuhang Yang, Yizhou Peng, Xionghu Zhong +2
This paper details the experimental results of adapting the OpenAI's Whisper model for Code-Switch Mandarin-English Speech Recognition (ASR) on the SEAME and ASRU2019 corpora. We c…
Intermediate-layer output Regularization for Attention-based Speech Recognition with Shared Decoder
Jicheng Zhang, Yizhou Peng, Haihua Xu +3
Intermediate layer output (ILO) regularization by means of multitask training on encoder side has been shown to be an effective approach to yielding improved results on a wide rang…
Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognition
Yizhou Peng, Yufei Liu, Jicheng Zhang +4
Internal Language Model Estimation (ILME) based language model (LM) fusion has been shown significantly improved recognition results over conventional shallow fusion in both intra-…
Minimum word error training for non-autoregressive Transformer-based code-switching ASR
Yizhou Peng, Jicheng Zhang, Haihua Xu +2
Non-autoregressive end-to-end ASR framework might be potentially appropriate for code-switching recognition task thanks to its inherent property that present output token being ind…
E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognition
Jicheng Zhang, Yizhou Peng, Pham Van Tung +3
In this paper, we propose a single multi-task learning framework to perform End-to-End (E2E) speech recognition (ASR) and accent recognition (AR) simultaneously. The proposed frame…
Multilingual Approach to Joint Speech and Accent Recognition with DNN-HMM Framework
Yizhou Peng, Jicheng Zhang, Haobo Zhang +3
Human can recognize speech, as well as the peculiar accent of the speech simultaneously. However, present state-of-the-art ASR system can rarely do that. In this paper, we propose…