6 citations · 16 across the 16 of their papers we have counts for
16 papers
EmoSURA: Towards Accurate Evaluation of Detailed and Long-Context Emotional Speech Captions
Xin Jing, Andreas Triantafyllopoulos, Jiadong Wang +3
Recent advancements in speech captioning models have enabled the generation of rich, fine-grained captions for emotional speech. However, the evaluation of such captions remains a…
CueNet: Robust Audio-Visual Speaker Extraction through Cross-Modal Cue Mining and Interaction
Jiadong Wang, Ke Zhang, Xinyuan Qian +3
Audio-visual speaker extraction has attracted increasing attention, as it removes the need for pre-registered speech and leverages the visual modality as a complement to audio. Alt…
SmoothCLAP: Soft-Target Enhanced Contrastive Language\--Audio Pretraining for Affective Computing
Xin Jing, Jiadong Wang, Andreas Triantafyllopoulos +4
The ambiguity of human emotions poses several challenges for machine learning models, as they often overlap and lack clear delineating boundaries. Contrastive language-audio pretra…
AV-SSAN: Audio-Visual Selective DoA Estimation through Explicit Multi-Band Semantic-Spatial Alignment
Yu Chen, Hongxu Zhu, Jiadong Wang +2
Audio-visual sound source localization (AV-SSL) estimates the position of sound sources by fusing auditory and visual cues. Current AV-SSL methodologies typically require spatially…
MELT: Towards Automated Multimodal Emotion Data Annotation by Leveraging LLM Embedded Knowledge
Xin Jing, Jiadong Wang, Iosif Tsangko +2
Although speech emotion recognition (SER) has advanced significantly with deep learning, annotation remains a major hurdle. Human annotation is not only costly but also subject to…
The First MPDD Challenge: Multimodal Personality-aware Depression Detection
Changzeng Fu, Zelin Fu, Qi Zhang +14
Depression is a widespread mental health issue affecting diverse age groups, with notable prevalence among college students and the elderly. However, existing datasets and detectio…
AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction
Wenxuan Wu, Xueyuan Chen, Shuai Wang +5
Audio-Visual Target Speaker Extraction (AV-TSE) aims to mimic the human ability to enhance auditory perception using visual cues. Although numerous models have been proposed recent…
Analytic Class Incremental Learning for Sound Source Localization with Privacy Protection
Xinyuan Qian, Xianghu Yue, Jiadong Wang +2
Sound Source Localization (SSL) enabling technology for applications such as surveillance and robotics. While traditional Signal Processing (SP)-based SSL methods provide analytic…
Human-Inspired Computing for Robust and Efficient Audio-Visual Speech Recognition
Qianhui Liu, Jiadong Wang, Yang Wang +3
Humans naturally perform audiovisual speech recognition (AVSR), enhancing the accuracy and robustness by integrating auditory and visual information. Spiking neural networks (SNNs)…
Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal
Yeying Jin, Xin Li, Jiadong Wang +2
Existing raindrop removal datasets have two shortcomings. First, they consist of images captured by cameras with a focus on the background, leading to the presence of blurry raindr…
Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention
Ruijie Tao, Xinyuan Qian, Yidi Jiang +3
Audio-visual target speaker extraction (AV-TSE) aims to extract the specific person's speech from the audio mixture given auxiliary visual cues. Previous methods usually search for…
Enhancing Real-World Active Speaker Detection with Multi-Modal Extraction Pre-Training
Ruijie Tao, Xinyuan Qian, Rohan Kumar Das +3
Audio-visual active speaker detection (AV-ASD) aims to identify which visible face is speaking in a scene with one or more persons. Most existing AV-ASD methods prioritize capturin…
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert
Jiadong Wang, Xinyuan Qian, Malu Zhang +2
Talking face generation, also known as speech-to-lip generation, reconstructs facial motions concerning lips given coherent speech input. The previous studies revealed the importan…
Predict-and-Update Network: Audio-Visual Speech Recognition Inspired by Human Speech Perception
Jiadong Wang, Xinyuan Qian, Haizhou Li
Audio and visual signals complement each other in human speech perception, so do they in speech recognition. The visual hint is less evident than the acoustic hint, but more robust…
Multi-target DoA Estimation with an Audio-visual Fusion Mechanism
Xinyuan Qian, Maulik Madhavi, Zexu Pan +2
Most of the prior studies in the spatial \ac{DoA} domain focus on a single modality. However, humans use auditory and visual senses to detect the presence of sound sources. With th…
Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks
Malu Zhang, Jiadong Wang, Burin Amornpaisannon +8
Spiking Neural Networks (SNNs) use spatio-temporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-po…