514 citations · 1.2k across the 41 of their papers we have counts for
41 papers
Leveraging Mamba with Full-Face Vision for Audio-Visual Speech Enhancement
Rong Chao, Wenze Ren, You-Jin Li +5
Recent Mamba-based models have shown promise in speech enhancement by efficiently modeling long-range temporal dependencies. However, models like Speech Enhancement Mamba (SEMamba)…
Linguistic Knowledge Transfer Learning for Speech Enhancement
Kuo-Hsuan Hung, Xugang Lu, Szu-Wei Fu +4
Linguistic knowledge plays a crucial role in spoken language comprehension. It provides essential semantic and syntactic context for speech perception in noisy environments. Howeve…
RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier
Pin-Yen Huang, Szu-Wei Fu, Yu Tsao
State-of-the-art (SOTA) semi-supervised learning techniques, such as FixMatch and it's variants, have demonstrated impressive performance in classification tasks. However, these me…
The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction
Wen-Chin Huang, Szu-Wei Fu, Erica Cooper +5
We present the third edition of the VoiceMOS Challenge, a scientific initiative designed to advance research into automatic prediction of human speech ratings. There were three tra…
Exploiting Consistency-Preserving Loss and Perceptual Contrast Stretching to Boost SSL-based Speech Enhancement
Muhammad Salman Khan, Moreno La Quatra, Kuo-Hsuan Hung +3
Self-supervised representation learning (SSL) has attained SOTA results on several downstream speech tasks, but SSL-based speech enhancement (SE) solutions still lag behind. To add…
DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +4
Recent speech language models (SLMs) typically incorporate pre-trained speech models to extend the capabilities from large language models (LLMs). In this paper, we propose a Descr…
An Investigation of Incorporating Mamba for Speech Enhancement
Rong Chao, Wen-Huang Cheng, Moreno La Quatra +4
This work aims to investigate the use of a recently proposed, attention-free, scalable state-space model (SSM), Mamba, for the speech enhancement (SE) task. In particular, we emplo…
Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
Szu-Wei Fu, Kuo-Hsuan Hung, Yu Tsao +1
Speech quality estimation has recently undergone a paradigm shift from human-hearing expert designs to machine-learning models. However, current models rely mainly on supervised le…
Multi-objective Non-intrusive Hearing-aid Speech Assessment Model
Hsin-Tien Chiang, Szu-Wei Fu, Hsin-Min Wang +2
Without the need for a clean reference, non-intrusive speech assessment methods have caught great attention for objective evaluations. While deep learning models have been used to…
A Study on Incorporating Whisper for Robust Speech Assessment
Ryandhimas E. Zezario, Yu-Wen Chen, Szu-Wei Fu +3
This research introduces an enhanced version of the multi-objective speech assessment model--MOSA-Net+, by leveraging the acoustic features from Whisper, a large-scaled weakly supe…
Study on the Correlation between Objective Evaluations and Subjective Speech Quality and Intelligibility
Hsin-Tien Chiang, Kuo-Hsuan Hung, Szu-Wei Fu +3
Subjective tests are the gold standard for evaluating speech quality and intelligibility; however, they are time-consuming and expensive. Thus, objective measures that align with h…
QuAVF: Quality-aware Audio-Visual Fusion for Ego4D Talking to Me Challenge
Hsi-Che Lin, Chien-Yi Wang, Min-Hung Chen +2
This technical report describes our QuAVF@NTU-NVIDIA submission to the Ego4D Talking to Me (TTM) Challenge 2023. Based on the observation from the TTM task and the provided dataset…
Improving Meeting Inclusiveness using Speech Interruption Analysis
Szu-Wei Fu, Yaran Fan, Yasaman Hosseinkashi +2
Meetings are a pervasive method of communication within all types of companies and organizations, and using remote collaboration systems to conduct meetings has increased dramatica…
Real-time Speech Interruption Analysis: From Cloud to Client Deployment
Quchen Fu, Szu-Wei Fu, Yaran Fan +4
Meetings are an essential form of communication for all types of organizations, and remote collaboration systems have been much more widely used since the COVID-19 pandemic. One ma…
Boosting Self-Supervised Embeddings for Speech Enhancement
Kuo-Hsuan Hung, Szu-wei Fu, Huan-Hsin Tseng +3
Self-supervised learning (SSL) representation for speech has achieved state-of-the-art (SOTA) performance on several downstream tasks. However, there remains room for improvement i…
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Ryandhimas E. Zezario, Szu-wei Fu, Fei Chen +3
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention. Many studies report that these DL-based models yield satisfactory assessme…
Perceptual Contrast Stretching on Target Feature for Speech Enhancement
Rong Chao, Cheng Yu, Szu-Wei Fu +2
Speech enhancement (SE) performance has improved considerably owing to the use of deep learning models as a base function. Herein, we propose a perceptual contrast stretching (PCS)…
OSSEM: one-shot speaker adaptive speech enhancement using meta learning
Cheng Yu, Szu-Wei Fu, Tsun-An Hsieh +2
Although deep learning (DL) has achieved notable progress in speech enhancement (SE), further research is still required for a DL-based SE system to adapt effectively and efficient…
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points
Yu-Chen Lin, Cheng Yu, Yi-Te Hsu +3
Numerous compression and acceleration strategies have achieved outstanding results on classification tasks in various fields, such as computer vision and speech signal processing.…
Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features
Ryandhimas E. Zezario, Szu-Wei Fu, Fei Chen +3
In this study, we propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously. Experimental…
MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech
Szu-Wei Fu, Cheng Yu, Kuo-Hsuan Hung +2
Most of the deep learning-based speech enhancement models are learned in a supervised manner, which implies that pairs of noisy and clean speech are required during training. Conse…
SpeechBrain: A General-Purpose Speech Toolkit
Mirco Ravanelli, Titouan Parcollet, Peter Plantinga +18
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, fle…
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
Szu-Wei Fu, Cheng Yu, Tsun-An Hsieh +4
The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory. Ob…
STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model
Ryandhimas E. Zezario, Szu-Wei Fu, Chiou-Shann Fuh +2
The calculation of most objective speech intelligibility assessment metrics requires clean speech as a reference. Such a requirement may limit the applicability of these metrics in…