116 citations · 146 across the 16 of their papers we have counts for
16 papers
Enhance audio generation controllability through representation similarity regularization
Yangyang Shi, Gael Le Lan, Varun Nagaraja +6
This paper presents an innovative approach to enhance control over audio generation by emphasizing the alignment between audio and text representations during model training. In th…
Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute
William Chen, Xuankai Chang, Yifan Peng +3
Self-supervised learning (SSL) has led to great strides in speech processing. However, the resources needed to train these models has become prohibitively large as they continue to…
Scaling Speech Technology to 1,000+ Languages
Vineel Pratap, Andros Tjandra, Bowen Shi +13
Expanding the language coverage of speech technology has the potential to improve access to information for many more people. However, current speech technology is restricted to ab…
Ripple sparse self-attention for monaural speech enhancement
Qiquan Zhang, Hongxu Zhu, Qi Song +3
The use of Transformer represents a recent success in speech enhancement. However, as its core component, self-attention suffers from quadratic complexity, which is computationally…
TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio
Anurag Kumar, Ke Tan, Zhaoheng Ni +4
Measuring quality and intelligibility of a speech signal is usually a critical step in development of speech processing systems. To enable this, a variety of metrics to measure qua…
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
Brian Yan, Jiatong Shi, Yun Tang +13
ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community. ESPnet-ST-v2 supports 1) offlin…
ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding
Yen-Ju Lu, Xuankai Chang, Chenda Li +10
This paper presents recent progress on integrating speech separation and enhancement (SSE) into the ESPnet toolkit. Compared with the previous ESPnet-SE work, numerous features hav…
Towards Low-distortion Multi-channel Speech Enhancement: The ESPNet-SE Submission to The L3DAS22 Challenge
Yen-Ju Lu, Samuele Cornell, Xuankai Chang +5
This paper describes our submission to the L3DAS22 Challenge Task 1, which consists of speech enhancement with 3D Ambisonic microphones. The core of our approach combines Deep Neur…
Time-Frequency Attention for Monaural Speech Enhancement
Qiquan Zhang, Qi Song, Zhaoheng Ni +2
Most studies on speech enhancement generally don't consider the energy distribution of speech in time-frequency (T-F) representation, which is important for accurate prediction of…
TorchAudio: Building Blocks for Audio and Speech Processing
Yao-Yuan Yang, Moto Hira, Zhaoheng Ni +20
This document describes version 0.10 of TorchAudio: building blocks for machine learning applications in the audio and speech processing domain. The objective of TorchAudio is to a…
Combining Spatial Clustering with LSTM Speech Models for Multichannel Speech Enhancement
Felix Grezes, Zhaoheng Ni, Viet Anh Trinh +1
Recurrent neural networks using the LSTM architecture can achieve significant single-channel noise reduction. It is not obvious, however, how to apply them to multi-channel inputs…
Improved MVDR Beamforming Using LSTM Speech Models to Clean Spatial Clustering Masks
Zhaoheng Ni, Felix Grezes, Viet Anh Trinh +1
Spatial clustering techniques can achieve significant multi-channel noise reduction across relatively arbitrary microphone configurations, but have difficulty incorporating a detai…
Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks
Felix Grezes, Zhaoheng Ni, Viet Anh Trinh +1
Recent works have shown that Deep Recurrent Neural Networks using the LSTM architecture can achieve strong single-channel speech enhancement by estimating time-frequency masks. How…
WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation
Zhaoheng Ni, Yong Xu, Meng Yu +4
This paper aims at eliminating the interfering speakers' speech, additive noise, and reverberation from the noisy multi-talker speech mixture that benefits automatic speech recogni…
Onssen: an open-source speech separation and enhancement library
Zhaoheng Ni, Michael I Mandel
Speech separation is an essential task for multi-talker speech recognition. Recently many deep learning approaches are proposed and have been constantly refreshing the state-of-the…
Mask-dependent Phase Estimation for Monaural Speaker Separation
Zhaoheng Ni, Michael I Mandel
Speaker separation refers to isolating speech of interest in a multi-talker environment. Most methods apply real-valued Time-Frequency (T-F) masks to the mixture Short-Time Fourier…