20 citations · 86 across the 25 of their papers we have counts for
25 papers
From Oracle to Noisy Context: Mitigating Contextual Exposure Bias in Speech-LLMs
Xiaoyong Guo, Nanjie Li, Zijie Zeng +4
Contextual automatic speech recognition (ASR) with Speech-LLMs is typically trained with oracle conversation history, but relies on error-prone history at inference, causing a trai…
Internal Language Model Estimation based Adaptive Language Model Fusion for Domain Adaptation
Rao Ma, Xiaobo Wu, Jin Qiu +4
ASR model deployment environment is ever-changing, and the incoming speech can be switched across different domains during a session. This brings a challenge for effective domain a…
Speech-text based multi-modal training with bidirectional attention for improved speech recognition
Yuhang Yang, Haihua Xu, Hao Huang +2
To let the state-of-the-art end-to-end ASR model enjoy data efficiency, as well as much more unpaired text data by multi-modal training, one needs to address two problems: 1) the s…
Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarization
Hexin Liu, Haihua Xu, Leibny Paola Garcia +3
Code-switching (CS) refers to the phenomenon that languages switch within a speech signal and leads to language confusion for automatic speech recognition (ASR). This paper aims to…
Random Utterance Concatenation Based Data Augmentation for Improving Short-video Speech Recognition
Yist Y. Lin, Tao Han, Haihua Xu +6
One of limitations in end-to-end automatic speech recognition (ASR) framework is its performance would be compromised if train-test utterance lengths are mismatched. In this paper,…
Intermediate-layer output Regularization for Attention-based Speech Recognition with Shared Decoder
Jicheng Zhang, Yizhou Peng, Haihua Xu +3
Intermediate layer output (ILO) regularization by means of multitask training on encoder side has been shown to be an effective approach to yielding improved results on a wide rang…
Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognition
Yizhou Peng, Yufei Liu, Jicheng Zhang +4
Internal Language Model Estimation (ILME) based language model (LM) fusion has been shown significantly improved recognition results over conventional shallow fusion in both intra-…
Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASR
Yufei Liu, Rao Ma, Haihua Xu +3
An end-to-end (E2E) ASR model implicitly learns a prior Internal Language Model (ILM) from the training transcripts. To fuse an external LM using Bayes posterior theory, the log li…
Minimum word error training for non-autoregressive Transformer-based code-switching ASR
Yizhou Peng, Jicheng Zhang, Haihua Xu +2
Non-autoregressive end-to-end ASR framework might be potentially appropriate for code-switching recognition task thanks to its inherent property that present output token being ind…
Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech
Duo Ma, Nana Hou, Van Tung Pham +2
To realize robust end-to-end Automatic Speech Recognition(E2E ASR) under radio communication condition, we propose a multitask-based method to joint train a Speech Enhancement (SE)…
E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognition
Jicheng Zhang, Yizhou Peng, Pham Van Tung +3
In this paper, we propose a single multi-task learning framework to perform End-to-End (E2E) speech recognition (ASR) and accent recognition (AR) simultaneously. The proposed frame…
Enriching Under-Represented Named-Entities To Improve Speech Recognition Performance
Tingzhi Mao, Yerbolat Khassanov, Van Tung Pham +4
Automatic speech recognition (ASR) for under-represented named-entity (UR-NE) is challenging due to such named-entities (NE) have insufficient instances and poor contextual coverag…
The NTU-AISG Text-to-speech System for Blizzard Challenge 2020
Haobo Zhang, Tingzhi Mao, Haihua Xu +1
We report our NTU-AISG Text-to-speech (TTS) entry systems for the Blizzard Challenge 2020 in this paper. There are two TTS tasks in this year's challenge, one is a Mandarin TTS tas…
Multilingual Approach to Joint Speech and Accent Recognition with DNN-HMM Framework
Yizhou Peng, Jicheng Zhang, Haobo Zhang +3
Human can recognize speech, as well as the peculiar accent of the speech simultaneously. However, present state-of-the-art ASR system can rarely do that. In this paper, we propose…
Monolingual Data Selection Analysis for English-Mandarin Hybrid Code-switching Speech Recognition
Haobo Zhang, Haihua Xu, Van Tung Pham +2
In this paper, we conduct data selection analysis in building an English-Mandarin code-switching (CS) speech recognition (CSSR) system, which is aimed for a real CSSR contest in Ch…
Leveraging Text Data Using Hybrid Transformer-LSTM Based End-to-End ASR in Transfer Learning
Zhiping Zeng, Van Tung Pham, Haihua Xu +4
In this work, we study leveraging extra text data to improve low-resource end-to-end ASR under cross-lingual transfer learning setting. To this end, we extend our prior work [1], a…
Approaches to Improving Recognition of Underrepresented Named Entities in Hybrid ASR Systems
Tingzhi Mao, Yerbolat Khassanov, Van Tung Pham +3
In this paper, we present a series of complementary approaches to improve the recognition of underrepresented named entities (NE) in hybrid ASR systems without compromising overall…
Spatial-Scale Aligned Network for Fine-Grained Recognition
Lizhao Gao, Haihua Xu, Chong Sun +2
Existing approaches for fine-grained visual recognition focus on learning marginal region-based representations while neglecting the spatial and scale misalignments, leading to inf…
Independent language modeling architecture for end-to-end ASR
Van Tung Pham, Haihua Xu, Yerbolat Khassanov +5
The attention-based end-to-end (E2E) automatic speech recognition (ASR) architecture allows for joint optimization of acoustic and language models within a single network. However,…
Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
Yerbolat Khassanov, Haihua Xu, Van Tung Pham +4
The lack of code-switch training data is one of the major concerns in the development of end-to-end code-switching automatic speech recognition (ASR) models. In this work, we propo…
Enriching Rare Word Representations in Neural Language Models by Embedding Matrix Augmentation
Yerbolat Khassanov, Zhiping Zeng, Van Tung Pham +2
The neural language models (NLM) achieve strong generalization capability by learning the dense representation of words and using them to estimate probability distribution function…
On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition
Zhiping Zeng, Yerbolat Khassanov, Van Tung Pham +3
Code-switching (CS) refers to a linguistic phenomenon where a speaker uses different languages in an utterance or between alternating utterances. In this work, we study end-to-end…
Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition
Pengcheng Guo, Haihua Xu, Lei Xie +1
In this paper, we present our overall efforts to improve the performance of a code-switching speech recognition system using semi-supervised training methods from lexicon learning…
Mandarin tone modeling using recurrent neural networks
Hao Huang, Ying Hu, Haihua Xu
We propose an Encoder-Classifier framework to model the Mandarin tones using recurrent neural networks (RNN). In this framework, extracted frames of features for tone classificatio…