62 citations · 247 across the 32 of their papers we have counts for
32 papers
MSBraM: A Multi-scale Self-supervised Brain Foundation Model for Hierarchical EEG Dynamics Learning
Tao Zhou, Jing Han, Lingyu Shu +1
Self-supervised foundation models have recently shown strong potential for electroencephalogram (EEG)-based analysis. However, existing approaches struggle to capture the inherentl…
CHARM: Charge Calibration and Acoustic Rescue for LLM-based Multimodal Sarcasm Detection
Qiyang Sun, Yi Chang, Yupei Li +3
Sarcasm detection, the identification of discrepancies between literal and intended meaning, is a fundamental task in affective computing. However, zero-shot instruction-tuned Larg…
SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition
Qiyang Sun, Yi Chang, Zixing Zhang +1
Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks o…
AudioFab: Building A General and Intelligent Audio Factory through Tool Learning
Cheng Zhu, Jing Han, Qianshuai Xue +3
Currently, artificial intelligence is profoundly transforming the audio domain; however, numerous advanced algorithms and tools remain fragmented, lacking a unified and efficient f…
Structured Prompting and LLM Ensembling for Multimodal Conversational Aspect-based Sentiment Analysis
Zhiqiang Gao, Shihao Gao, Zixing Zhang +3
Understanding sentiment in multimodal conversations is a complex yet crucial challenge toward building emotionally intelligent AI systems. The Multimodal Conversational Aspect-base…
PTalker: Personalized Speech-Driven 3D Talking Head Animation via Style Disentanglement and Modality Alignment
Bin Wang, Yang Xu, Huan Zhao +2
Speech-driven 3D talking head generation aims to produce lifelike facial animations precisely synchronized with speech. While considerable progress has been made in achieving high…
Tyee: A Unified, Modular, and Fully-Integrated Configurable Toolkit for Intelligent Physiological Health Care
Tao Zhou, Lingyu Shu, Zixing Zhang +1
Deep learning has shown great promise in physiological signal analysis, yet its progress is hindered by heterogeneous data formats, inconsistent preprocessing strategies, fragmente…
Foundation Model-based Evaluation of Neuropsychiatric Disorders: A Lifespan-Inclusive, Multi-Modal, and Multi-Lingual Study
Zhongren Dong, Haotian Guo, Weixiang Xu +2
Neuropsychiatric disorders, such as Alzheimer's disease (AD), depression, and autism spectrum disorder (ASD), are characterized by linguistic and acoustic abnormalities, offering p…
SACodec: Asymmetric Quantization with Semantic Anchoring for Low-Bitrate High-Fidelity Neural Speech Codecs
Zhongren Dong, Bin Wang, Jing Han +4
Neural Speech Codecs face a fundamental trade-off at low bitrates: preserving acoustic fidelity often compromises semantic richness. To address this, we introduce SACodec, a novel…
Pioneering Multimodal Emotion Recognition in the Era of Large Models: From Closed Sets to Open Vocabularies
Jing Han, Zhiqiang Gao, Shihao Gao +4
Recent advances in multimodal large language models (MLLMs) have demonstrated remarkable multi- and cross-modal integration capabilities. However, their potential for fine-grained…
DEBATE: A Dataset for Disentangling Textual Ambiguity in Mandarin Through Speech
Haotian Guo, Jing Han, Yongfeng Tu +5
Despite extensive research on textual and visual disambiguation, disambiguation through speech (DTS) remains underexplored. This is largely due to the lack of high-quality datasets…
Celler:A Genomic Language Model for Long-Tailed Single-Cell Annotation
Huan Zhao, Yiming Liu, Jina Yao +3
Recent breakthroughs in single-cell technology have ushered in unparalleled opportunities to decode the molecular intricacy of intricate biological systems, especially those linked…
ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesis
Xiangheng He, Junjie Chen, Zixing Zhang +1
Prosody contains rich information beyond the literal meaning of words, which is crucial for the intelligibility of speech. Current models still fall short in phrasing and intonatio…
ParaLBench: A Large-Scale Benchmark for Computational Paralinguistics over Acoustic Foundation Models
Zixing Zhang, Weixiang Xu, Zhongren Dong +5
Computational paralinguistics (ComParal) aims to develop algorithms and models to automatically detect, analyze, and interpret non-verbal information from speech communication, e.…
Re-Parameterization of Lightweight Transformer for On-Device Speech Emotion Recognition
Zixing Zhang, Zhongren Dong, Weixiang Xu +1
With the increasing implementation of machine learning models on edge or Internet-of-Things (IoT) devices, deploying advanced models on resource-constrained IoT devices remains cha…
Semi-Supervised Cognitive State Classification from Speech with Multi-View Pseudo-Labeling
Yuanchao Li, Zixing Zhang, Jing Han +2
The lack of labeled data is a common challenge in speech classification tasks, particularly those requiring extensive subjective assessment, such as cognitive state classification.…
ESIHGNN: Event-State Interactions Infused Heterogeneous Graph Neural Network for Conversational Emotion Recognition
Xupeng Zha, Huan Zhao, Zixing Zhang
Conversational Emotion Recognition (CER) aims to predict the emotion expressed by an utterance (referred to as an ``event'') during a conversation. Existing graph-based methods mai…
Adaptive Speech Emotion Representation Learning Based On Dynamic Graph
Yingxue Gao, Huan Zhao, Zixing Zhang
Graph representation learning has become a hot research topic due to its powerful nonlinear fitting capability in extracting representative node embeddings. However, for sequential…
Intelligent Cardiac Auscultation for Murmur Detection via Parallel-Attentive Models with Uncertainty Estimation
Zixing Zhang, Tao Pang, Jing Han +1
Heart murmurs are a common manifestation of cardiovascular diseases and can provide crucial clues to early cardiac abnormalities. While most current research methods primarily focu…
HAFFormer: A Hierarchical Attention-Free Framework for Alzheimer's Disease Detection From Spontaneous Speech
Zhongren Dong, Zixing Zhang, Weixiang Xu +3
Automatically detecting Alzheimer's Disease (AD) from spontaneous speech plays an important role in its early diagnosis. Recent approaches highly rely on the Transformer architectu…
STAA-Net: A Sparse and Transferable Adversarial Attack for Speech Emotion Recognition
Yi Chang, Zhao Ren, Zixing Zhang +6
Speech contains rich information on the emotions of humans, and Speech Emotion Recognition (SER) has been an important topic in the area of human-computer interaction. The robustne…
Customising General Large Language Models for Specialised Emotion Recognition Tasks
Liyizhe Peng, Zixing Zhang, Tao Pang +4
The advent of large language models (LLMs) has gained tremendous attention over the past year. Previous studies have shown the astonishing performance of LLMs not only in other tas…
Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models
Zixing Zhang, Liyizhe Peng, Tao Pang +3
After the inception of emotion recognition or affective computing, it has increasingly become an active research topic due to its broad applications. Over the past couple of decade…
Dynamic Restrained Uncertainty Weighting Loss for Multitask Learning of Vocal Expression
Meishu Song, Zijiang Yang, Andreas Triantafyllopoulos +6
We propose a novel Dynamic Restrained Uncertainty Weighting Loss to experimentally handle the problem of balancing the contributions of multiple tasks on the ICML ExVo 2022 Challen…