2 citations · 6 across the 17 of their papers we have counts for
17 papers
Optimal Transport-based Semantic Alignment for LLM-based Audio-Visual Speech Recognition
Xugang Lu, Peng Shen, Yu Tsao +1
Large language model (LLM)-based audio-visual speech recognition (LLM-AVSR) has recently demonstrated strong robustness in adverse acoustic environments by leveraging complementary…
New Insights into Optimal Alignment of Acoustic and Linguistic Representations for Knowledge Transfer in ASR
Xugang Lu, Peng Shen, Hisashi Kawai
Aligning acoustic and linguistic representations is a central challenge to bridge the pre-trained models in knowledge transfer for automatic speech recognition (ASR). This alignmen…
Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR
Xugang Lu, Peng Shen, Yu Tsao +1
Transferring linguistic knowledge from a pretrained language model (PLM) to acoustic feature learning has proven effective in enhancing end-to-end automatic speech recognition (E2E…
Retrieval-Augmented Speech Recognition Approach for Domain Challenges
Peng Shen, Xugang Lu, Hisashi Kawai
Speech recognition systems often face challenges due to domain mismatch, particularly in real-world applications where domain-specific data is unavailable because of data accessibi…
Temporal Order Preserved Optimal Transport-based Cross-modal Knowledge Transfer Learning for ASR
Xugang Lu, Peng Shen, Yu Tsao +1
Transferring linguistic knowledge from a pretrained language model (PLM) to an acoustic model has been shown to greatly improve the performance of automatic speech recognition (ASR…
Generative linguistic representation for spoken language identification
Peng Shen, Xuguang Lu, Hisashi Kawai
Effective extraction and application of linguistic features are central to the enhancement of spoken Language IDentification (LID) performance. With the success of recent large mod…
Speaker Mask Transformer for Multi-talker Overlapped Speech Recognition
Peng Shen, Xugang Lu, Hisashi Kawai
Multi-talker overlapped speech recognition remains a significant challenge, requiring not only speech recognition but also speaker diarization tasks to be addressed. In this paper,…
Neural domain alignment for spoken language recognition based on optimal transport
Xugang Lu, Peng Shen, Yu Tsao +1
Domain shift poses a significant challenge in cross-domain spoken language recognition (SLR) by reducing its effectiveness. Unsupervised domain adaptation (UDA) algorithms have bee…
Hierarchical Cross-Modality Knowledge Transfer with Sinkhorn Attention for CTC-based ASR
Xugang Lu, Peng Shen, Yu Tsao +1
Due to the modality discrepancy between textual and acoustic modeling, efficiently transferring linguistic knowledge from a pretrained language model (PLM) to acoustic encoding for…
Cross-modal Alignment with Optimal Transport for CTC-based ASR
Xugang Lu, Peng Shen, Yu Tsao +1
Temporal connectionist temporal classification (CTC)-based automatic speech recognition (ASR) is one of the most successful end to end (E2E) ASR frameworks. However, due to the tok…
Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition
Peng Shen, Xugang Lu, Hisashi Kawai
For Mandarin end-to-end (E2E) automatic speech recognition (ASR) tasks, compared to character-based modeling units, pronunciation-based modeling units could improve the sharing of…
Transducer-based language embedding for spoken language identification
Peng Shen, Xugang Lu, Hisashi Kawai
The acoustic and linguistic features are important cues for the spoken language identification (LID) task. Recent advanced LID systems mainly use acoustic features that lack the us…
Partial Coupling of Optimal Transport for Spoken Language Identification
Xugang Lu, Peng Shen, Yu Tsao +1
In order to reduce domain discrepancy to improve the performance of cross-domain spoken language identification (SLID) system, as an unsupervised domain adaptation (UDA) method, we…
Siamese Neural Network with Joint Bayesian Model Structure for Speaker Verification
Xugang Lu, Peng Shen, Yu Tsao +1
Generative probability models are widely used for speaker verification (SV). However, the generative models are lack of discriminative feature selection ability. As a hypothesis te…
Coupling a generative model with a discriminative learning framework for speaker verification
Xugang Lu, Peng Shen, Yu Tsao +1
The speaker verification (SV) task is to decide whether an utterance is spoken by a target or an imposter speaker. For most studies, a log-likelihood ratio (LLR) score is estimated…
Unsupervised neural adaptation model based on optimal transport for spoken language identification
Xugang Lu, Peng Shen, Yu Tsao +1
Due to the mismatch of statistical distributions of acoustic speech between training and testing sets, the performance of spoken language identification (SLID) could be drastically…
Cross-scale Attention Model for Acoustic Event Classification
Xugang Lu, Peng Shen, Sheng Li +2
A major advantage of a deep convolutional neural network (CNN) is that the focused receptive field size is increased by stacking multiple convolutional layers. Accordingly, the mod…