36 citations · 50 across the 13 of their papers we have counts for
14 papers
Multimodal Target Speaker Extraction: Towards Unified Speaker Cues Across Modalities
Xinyuan Qian, Yanghao Zhou, Ziyang Jiang +10
Target Speaker Extraction (TSE) is pivotal in speech communication and human-computer interaction, enabling the isolation of a specific speaker's voice from complex acoustic enviro…
CHARM: Charge Calibration and Acoustic Rescue for LLM-based Multimodal Sarcasm Detection
Qiyang Sun, Yi Chang, Yupei Li +3
Sarcasm detection, the identification of discrepancies between literal and intended meaning, is a fundamental task in affective computing. However, zero-shot instruction-tuned Larg…
SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition
Qiyang Sun, Yi Chang, Zixing Zhang +1
Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks o…
AffectSpeech: A Large-Scale Emotional Speech Dataset with Fine-Grained Textual Descriptions for Speech Emotion Captioning and Synthesis
Tianhua Qi, Wenming Zheng, Björn W. Schuller +2
Emotion is essential in spoken communication, yet most existing frameworks in speech emotion modeling rely on predefined categories or low-dimensional continuous attributes, which…
CueNet: Robust Audio-Visual Speaker Extraction through Cross-Modal Cue Mining and Interaction
Jiadong Wang, Ke Zhang, Xinyuan Qian +3
Audio-visual speaker extraction has attracted increasing attention, as it removes the need for pre-registered speech and leverages the visual modality as a complement to audio. Alt…
Cross-Dialect Bird Species Recognition with Dialect-Calibrated Augmentation
Jiani Ding, Qiyang Sun, Alican Akman +1
Dialect variation hampers automatic recognition of bird calls collected by passive acoustic monitoring. We address the problem on DB3V, a three-region, ten-species corpus of 8-s cl…
Audio-based Kinship Verification Using Age Domain Conversion
Qiyang Sun, Alican Akman, Xin Jing +2
Audio-based kinship verification (AKV) is important in many domains, such as home security monitoring, forensic identification, and social network analysis. A key challenge in the…
Audio Explanation Synthesis with Generative Foundation Models
Alican Akman, Qiyang Sun, Björn W. Schuller
The increasing success of audio foundation models across various tasks has led to a growing need for improved interpretability to understand their intricate decision-making process…
Synthia's Melody: A Benchmark Framework for Unsupervised Domain Adaptation in Audio
Chia-Hsin Lin, Charles Jones, Björn W. Schuller +1
Despite significant advancements in deep learning for vision and natural language, unsupervised domain adaptation in audio remains relatively unexplored. We, in part, attribute thi…
Data Augmentation for Dementia Detection in Spoken Language
Anna Hlédiková, Dominika Woszczyk, Alican Akman +2
Dementia is a growing problem as our society ages, and detection methods are often invasive and expensive. Recent deep-learning techniques can offer a faster diagnosis and have sho…
The ACII 2022 Affective Vocal Bursts Workshop & Competition: Understanding a critically understudied modality of emotional expression
Alice Baird, Panagiotis Tzirakis, Jeffrey A. Brooks +5
The ACII Affective Vocal Bursts Workshop & Competition is focused on understanding multiple affective dimensions of vocal bursts: laughs, gasps, cries, screams, and many other non-…
SVTS: Scalable Video-to-Speech Synthesis
Rodrigo Mira, Alexandros Haliassos, Stavros Petridis +2
Video-to-speech synthesis (also known as lip-to-speech) refers to the translation of silent lip movements into the corresponding audio. This task has received an increasing amount…
The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Bursts
Alice Baird, Panagiotis Tzirakis, Gauthier Gidel +7
The ICML Expressive Vocalization (ExVo) Competition is focused on understanding and generating vocal bursts: laughs, gasps, cries, and other non-verbal vocalizations that are centr…
HEAR: Holistic Evaluation of Audio Representations
Joseph Turian, Jordie Shier, Humair Raj Khan +20
What audio embedding approach generalizes best to a wide range of downstream tasks across a variety of everyday domains without fine-tuning? The aim of the HEAR benchmark is to dev…