24 citations · 99 across the 27 of their papers we have counts for
28 papers
Speaker Tracking: Segment-online Multi-talker Organization with a Varying Number of Speakers
Vahid A. Kalkhorani, Daniel Wong, Jacob Donley +3
Speaker tracking is the task of separating and following multiple speakers over time. It must address overlapped speech, speech onset and offset, talker identity, and time-varying…
CHiME-9 ECHI: A Machine Learning Challenge for Enhancing Conversations to Address Hearing Impairment
Robert Sutherland, Thomas Kuebert, Marko Lugger +6
This work presents the task and results of the CHiME-9 challenge for Enhancing Conversations to address Hearing Impairment. The challenge considers the scenario of four-party conve…
Cloud-Boosted Low-Compute Multi-Channel Speech Enhancement
Xulin Fan, Juan Azcarreta, Ashutosh Pandey +5
Low-latency, low-compute speech enhancement is essential for wearable devices with real-time communication requirements, but strict computational constraints significantly limit on…
Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement
Dongheon Lee, Ashutosh Pandey, Sanjeel Parekh +4
While the spatial directivity of multichannel speech enhancement algorithms improves with the number of microphones, fitting large capture arrays into real-world edge devices is ty…
Towards Localizing Conversation Partners using Head Motion
Payal Mohapatra, Calvin Murdock, Ali Aroudi +4
Many individuals struggle to understand conversation partners in noisy settings, particularly amid background speakers or due to hearing impairments. Emerging wearables like smartg…
FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses
Zhongweiyang Xu, Ali Aroudi, Ke Tan +4
This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart…
All Neural Low-latency Directional Speech Extraction
Ashutosh Pandey, Sanha Lee, Juan Azcarreta +2
We introduce a novel all neural model for low-latency directional speech extraction. The model uses direction of arrival (DOA) embeddings from a predefined spatial grid, which are…
AV-CrossNet: an Audiovisual Complex Spectral Mapping Network for Speech Separation By Leveraging Narrow- and Cross-Band Modeling
Vahid Ahmadi Kalkhorani, Cheng Yu, Anurag Kumar +3
Adding visual cues to audio-based speech separation can improve separation performance. This paper introduces AV-CrossNet, an audiovisual (AV) system for speech enhancement, target…
A Closer Look at Wav2Vec2 Embeddings for On-Device Single-Channel Speech Enhancement
Ravi Shankar, Ke Tan, Buye Xu +1
Self-supervised learned models have been found to be very effective for certain speech tasks such as automatic speech recognition, speaker identification, keyword spotting and othe…
On the Importance of Neural Wiener Filter for Resource Efficient Multichannel Speech Enhancement
Tsun-An Hsieh, Jacob Donley, Daniel Wong +2
We introduce a time-domain framework for efficient multichannel speech enhancement, emphasizing low latency and computational efficiency. This framework incorporates two compact de…
Decoupled Spatial and Temporal Processing for Resource Efficient Multichannel Speech Enhancement
Ashutosh Pandey, Buye Xu
We present a novel model designed for resource-efficient multichannel speech enhancement in the time domain, with a focus on low latency, lightweight, and low computational require…
TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio
Anurag Kumar, Ke Tan, Zhaoheng Ni +4
Measuring quality and intelligibility of a speech signal is usually a critical step in development of speech processing systems. To enable this, a variety of metrics to measure qua…
Rethinking complex-valued deep neural networks for monaural speech enhancement
Haibin Wu, Ke Tan, Buye Xu +2
Despite multiple efforts made towards adopting complex-valued deep neural networks (DNNs), it remains an open question whether complex-valued DNNs are generally more effective than…
LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders
Rodrigo Mira, Buye Xu, Jacob Donley +4
Audio-visual speech enhancement aims to extract clean speech from a noisy environment by leveraging not only the audio itself but also the target speaker's lip movements. This appr…
Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement
Kuan-Lin Chen, Daniel D. E. Wong, Ke Tan +3
Most speech enhancement (SE) models learn a point estimate and do not make use of uncertainty estimation in the learning process. In this paper, we show that modeling heteroscedast…
Spatially Selective Active Noise Control Systems
Tong Xiao, Buye Xu, Chuming Zhao
Active noise control (ANC) systems are commonly designed to achieve maximal sound reduction regardless of the incident direction of the sound. When desired sound is present, the st…
SAQAM: Spatial Audio Quality Assessment Metric
Pranay Manocha, Anurag Kumar, Buye Xu +4
Audio quality assessment is critical for assessing the perceptual realism of sounds. However, the time and expense of obtaining ''gold standard'' human judgments limit the availabi…
RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing
Efthymios Tzinis, Yossi Adi, Vamsi Krishna Ithapu +3
We present RemixIT, a simple yet effective self-supervised method for training speech enhancement without the need of a single isolated in-domain speech nor a noise waveform. Our a…
NICE-Beam: Neural Integrated Covariance Estimators for Time-Varying Beamformers
Jonah Casebeer, Jacob Donley, Daniel Wong +2
Estimating a time-varying spatial covariance matrix for a beamforming algorithm is a challenging task, especially for wearable devices, as the algorithm must compensate for time-va…
NORESQA: A Framework for Speech Quality Assessment using Non-Matching References
Pranay Manocha, Buye Xu, Anurag Kumar
The perceptual task of speech quality assessment (SQA) is a challenging task for machines to do. Objective SQA methods that rely on the availability of the corresponding clean refe…
Multichannel Speech Enhancement without Beamforming
Asutosh Pandey, Buye Xu, Anurag Kumar +3
Deep neural networks are often coupled with traditional spatial filters, such as MVDR beamformers for effectively exploiting spatial information. Even though single-stage end-to-en…
Time-domain Ad-hoc Array Speech Enhancement Using a Triple-path Network
Ashutosh Pandey, Buye Xu, Anurag Kumar +3
Deep neural networks (DNNs) are very effective for multichannel speech enhancement with fixed array geometries. However, it is not trivial to use DNNs for ad-hoc arrays with unknow…
TPARN: Triple-path Attentive Recurrent Network for Time-domain Multichannel Speech Enhancement
Ashutosh Pandey, Buye Xu, Anurag Kumar +3
In this work, we propose a new model called triple-path attentive recurrent network (TPARN) for multichannel speech enhancement in the time domain. TPARN extends a single-channel d…
Continual self-training with bootstrapped remixing for speech enhancement
Efthymios Tzinis, Yossi Adi, Vamsi K. Ithapu +2
We propose RemixIT, a simple and novel self-supervised training method for speech enhancement. The proposed method is based on a continuously self-training scheme that overcomes li…