9 papers
Speaker Tracking: Segment-online Multi-talker Organization with a Varying Number of Speakers
Vahid A. Kalkhorani, Daniel Wong, Jacob Donley +3
Speaker tracking is the task of separating and following multiple speakers over time. It must address overlapped speech, speech onset and offset, talker identity, and time-varying…
Cloud-Boosted Low-Compute Multi-Channel Speech Enhancement
Xulin Fan, Juan Azcarreta, Ashutosh Pandey +5
Low-latency, low-compute speech enhancement is essential for wearable devices with real-time communication requirements, but strict computational constraints significantly limit on…
Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement
Dongheon Lee, Ashutosh Pandey, Sanjeel Parekh +4
While the spatial directivity of multichannel speech enhancement algorithms improves with the number of microphones, fitting large capture arrays into real-world edge devices is ty…
Controlling the Parameterized Multi-channel Wiener Filter using a tiny neural network
Eric Grinstein, Ashutosh Pandey, Cole Li +6
Noise suppression and speech distortion are two important aspects to be balanced when designing multi-channel Speech Enhancement (SE) algorithms. Although neural network models hav…
Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
Joanna Hong, Sanjeel Parekh, Honglie Chen +4
Building reliable speech systems often requires combining multiple modalities, like audio and visual cues. While such multimodal solutions frequently lead to improvements in perfor…
M-BEST-RQ: A Multi-Channel Speech Foundation Model for Smart Glasses
Yufeng Yang, Desh Raj, Ju Lin +8
The growing popularity of multi-channel wearable devices, such as smart glasses, has led to a surge of applications such as targeted speech recognition and enhanced hearing. Howeve…
Spherical World-Locking for Audio-Visual Localization in Egocentric Videos
Heeseung Yun, Ruohan Gao, Ishwarya Ananthabhotla +6
Egocentric videos provide comprehensive contexts for user and scene understanding, spanning multisensory perception to behavioral interaction. We propose Spherical World-Locking (S…
On the Importance of Neural Wiener Filter for Resource Efficient Multichannel Speech Enhancement
Tsun-An Hsieh, Jacob Donley, Daniel Wong +2
We introduce a time-domain framework for efficient multichannel speech enhancement, emphasizing low latency and computational efficiency. This framework incorporates two compact de…
Multi-Channel Speech Enhancement using Graph Neural Networks
Panagiotis Tzirakis, Anurag Kumar, Jacob Donley
Multi-channel speech enhancement aims to extract clean speech from a noisy mixture using signals captured from multiple microphones. Recently proposed methods tackle this problem b…