262 citations · 1.4k across the 109 of their papers we have counts for
110 papers
The ISCSLP 2026 Real-World Audio-Visual Speech Enhancement Challenge: A Benchmark with Natural Mixtures and Degraded Video
Kai Li, Wenze Ren, Junjie Li +11
Audio-visual speech enhancement (AVSE) uses a target speaker's visible articulation to recover that speaker's voice from overlapped speech. Yet most evaluation protocols rely on sy…
Grounded Decoding for Autoregressive Speech Enhancement via Adaptive Code-Space Grounding and Local LLM Refinement
Hao Shi, Yuan Gao, Zhaoheng Ni +4
Large language model (LLM)-based autoregressive speech enhancement (SE) produces natural speech using learned clean-speech priors, but may hallucinate content unsupported by the in…
End-to-End Neural Decomposition with Koopman Operators for Time-Series Forecasting
De-Yan Lu, Xugang Lu, Yu Tsao +1
Koopman theory offers a linear-operator view of nonlinear sequence dynamics by lifting observations into a space where evolution is governed by a linear time-invariant Koopman oper…
IANS: Intelligibility-aware Null-steering Beamforming for Dual-Microphone Arrays
Wen-Yuan Ting, Syu-Siang Wang, Yu Tsao +1
Beamforming techniques are popular in speech-related applications due to their effective spatial filtering capabilities. Nonetheless, conventional beamforming techniques generally…
Mandarin Electrolaryngeal Speech Voice Conversion using Cross-domain Features
Hsin-Hao Chen, Yung-Lun Chien, Ming-Chi Yen +4
Patients who have had their entire larynx removed, including the vocal folds, owing to throat cancer may experience difficulties in speaking. In such cases, electrolarynx devices a…
BASPRO: a balanced script producer for speech corpus collection based on the genetic algorithm
Yu-Wen Chen, Hsin-Min Wang, Yu Tsao
The performance of speech-processing models is heavily influenced by the speech corpus that is used for training and evaluation. In this study, we propose BAlanced Script PROducer…
T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5
Chan-Jan Hsu, Ho-Lam Chung, Hung-yi Lee +1
In Spoken language understanding (SLU), a natural solution is concatenating pre-trained speech models (e.g. HuBERT) and pretrained language models (PLM, e.g. T5). Most previous wor…
CasNet: Investigating Channel Robustness for Speech Separation
Fan-Lin Wang, Yao-Fei Cheng, Hung-Shin Lee +2
Recording channel mismatch between training and testing conditions has been shown to be a serious problem for speech separation. This situation greatly reduces the separation perfo…
A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech
Li-Wei Chen, Yao-Fei Cheng, Hung-Shin Lee +2
The lack of clean speech is a practical challenge to the development of speech enhancement systems, which means that there is an inevitable mismatch between their training criterio…
Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
Yin-Ping Cho, Yu Tsao, Hsin-Min Wang +1
Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores. To accomplish end-to-end SVS effectively and efficiently, this wor…
ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding
Yen-Ju Lu, Xuankai Chang, Chenda Li +10
This paper presents recent progress on integrating speech separation and enhancement (SSE) into the ESPnet toolkit. Compared with the previous ESPnet-SE work, numerous features hav…
NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling
Chi-Chang Lee, Cheng-Hung Hu, Yu-Chen Lin +3
For deep learning-based speech enhancement (SE) systems, the training-test acoustic mismatch can cause notable performance degradation. To address the mismatch issue, numerous nois…
EPG2S: Speech Generation and Speech Enhancement based on Electropalatography and Audio Signals using Multimodal Learning
Li-Chin Chen, Po-Hsun Chen, Richard Tzong-Han Tsai +1
Speech generation and enhancement based on articulatory movements facilitate communication when the scope of verbal communication is absent, e.g., in patients who have lost the abi…
Filter-based Discriminative Autoencoders for Children Speech Recognition
Chiang-Lin Tai, Hung-Shin Lee, Yu Tsao +1
Children speech recognition is indispensable but challenging due to the diversity of children's speech. In this paper, we propose a filter-based discriminative autoencoder for acou…
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding
Chan-Jan Hsu, Hung-yi Lee, Yu Tsao
Transformer-based models are widely used in natural language understanding (NLU) tasks, and multimodal transformers have been effective in visual-language tasks. This study explore…
A Study of Using Cepstrogram for Countermeasure Against Replay Attacks
Shih-Kuang Lee, Yu Tsao, Hsin-Min Wang
This study investigated the cepstrogram properties and demonstrated their effectiveness as powerful countermeasures against replay attacks. A cepstrum analysis of replay attacks su…
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Ryandhimas E. Zezario, Szu-wei Fu, Fei Chen +3
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention. Many studies report that these DL-based models yield satisfactory assessme…
MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids
Ryandhimas E. Zezario, Fei Chen, Chiou-Shann Fuh +2
Improving the user's hearing ability to understand speech in noisy environments is critical to the development of hearing aid (HA) devices. For this, it is important to derive a me…
Partial Coupling of Optimal Transport for Spoken Language Identification
Xugang Lu, Peng Shen, Yu Tsao +1
In order to reduce domain discrepancy to improve the performance of cross-domain spoken language identification (SLID) system, as an unsupervised domain adaptation (UDA) method, we…
Subspace-based Representation and Learning for Phonotactic Spoken Language Recognition
Hung-Shin Lee, Yu Tsao, Shyh-Kang Jeng +1
Phonotactic constraints can be employed to distinguish languages by representing a speech utterance as a multinomial distribution or phone events. In the present study, we propose…
Perceptual Contrast Stretching on Target Feature for Speech Enhancement
Rong Chao, Cheng Yu, Szu-Wei Fu +2
Speech enhancement (SE) performance has improved considerably owing to the use of deep learning models as a base function. Herein, we propose a perceptual contrast stretching (PCS)…
Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks
Fan-Lin Wang, Hung-Shin Lee, Yu Tsao +1
Because the performance of speech separation is excellent for speech in which two speakers completely overlap, research attention has been shifted to dealing with more realistic sc…
Multi-target Extractor and Detector for Unknown-number Speaker Diarization
Chin-Yi Cheng, Hung-Shin Lee, Yu Tsao +1
Strong representations of target speakers can help extract important information about speakers and detect corresponding temporal regions in multi-speaker conversations. In this st…
Speech-enhanced and Noise-aware Networks for Robust Speech Recognition
Hung-Shin Lee, Pin-Yuan Chen, Yao-Fei Cheng +2
Compensation for channel mismatch and noise interference is essential for robust automatic speech recognition. Enhanced speech has been introduced into the multi-condition training…