132 citations · 189 across the 21 of their papers we have counts for
11 papers · 1 filter
NeuroHeed: Neuro-Steered Speaker Extraction using EEG Signals
Zexu Pan, Marvin Borsdorf, Siqi Cai +2
Humans possess the remarkable ability to selectively attend to a single speaker amidst competing voices and background noise, known as selective auditory attention. Recent studies…
Target Active Speaker Detection with Audio-visual Cues
Yidi Jiang, Ruijie Tao, Zexu Pan +1
In active speaker detection (ASD), we would like to detect whether an on-screen person is speaking based on audio-visual cues. Previous studies have primarily focused on modeling a…
Late Audio-Visual Fusion for In-The-Wild Speaker Diarization
Zexu Pan, Gordon Wichern, François G. Germain +2
Speaker diarization is well studied for constrained audios but little explored for challenging in-the-wild videos, which have more speakers, shorter utterances, and inconsistent on…
ImagineNET: Target Speaker Extraction with Intermittent Visual Cue through Embedding Inpainting
Zexu Pan, Wupeng Wang, Marvin Borsdorf +1
The speaker extraction technique seeks to single out the voice of a target speaker from the interfering voices in a speech mixture. Typically an auxiliary reference of the target s…
A Hybrid Continuity Loss to Reduce Over-Suppression for Time-domain Target Speaker Extraction
Zexu Pan, Meng Ge, Haizhou Li
The speaker extraction algorithm extracts the target speech from a mixture speech containing interference speech and background noise. The extraction process sometimes over-suppres…
Speaker Extraction with Co-Speech Gestures Cue
Zexu Pan, Xinyuan Qian, Haizhou Li
Speaker extraction seeks to extract the clean speech of a target speaker from a multi-talker mixture speech. There have been studies to use a pre-recorded speech sample or face ima…
USEV: Universal Speaker Extraction with Visual Cue
Zexu Pan, Meng Ge, Haizhou Li
A speaker extraction algorithm seeks to extract the target speaker's speech from a multi-talker speech mixture. The prior studies focus mostly on speaker extraction from a highly o…
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
Ruijie Tao, Zexu Pan, Rohan Kumar Das +3
Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers. The successful ASD depends on accurate interpretation of short-term and lo…
Selective Listening by Synchronizing Speech with Lips
Zexu Pan, Ruijie Tao, Chenglin Xu +1
A speaker extraction algorithm seeks to extract the speech of a target speaker from a multi-talker speech mixture when given a cue that represents the target speaker, such as a pre…
Muse: Multi-modal target speaker extraction with visual cues
Zexu Pan, Ruijie Tao, Chenglin Xu +1
Speaker extraction algorithm relies on the speech sample from the target speaker as the reference point to focus its attention. Such a reference speech is typically pre-recorded. O…
Multi-modal Attention for Speech Emotion Recognition
Zexu Pan, Zhaojie Luo, Jichen Yang +1
Emotion represents an essential aspect of human speech that is manifested in speech prosody. Speech, visual, and textual cues are complementary in human communication. In this pape…