167 citations · 215 across the 22 of their papers we have counts for
22 papers
Uncovering Latent Depression Severity for Binary Depression Detection via Advantage-weighting Ranking
Manning Gao, Tingyi Liu, Leheng Zhang +3
Automatic depression detection using audio-visual data faces significant challenges, particularly in disentangling overlapping feature distributions and establishing robust decisio…
Evaluating the Expressive Appropriateness of Speech in Rich Contexts
Tianrui Wang, Ziyang Ma, Yizhou Peng +26
Evaluating expressive speech remains challenging, as existing methods mainly assess emotional intensity and overlook whether a speech sample is expressively appropriate for its con…
Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Data
Sijie Mai, Shiqin Han, Haifeng Hu
Multimodal data encountered in real-world scenarios are typically of low quality, with noisy modalities and missing modalities being typical forms that severely hinder model perfor…
CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning
Ronghao Lin, Qiaolin He, Sijie Mai +5
Multimodal machine learning, mimicking the human brain's ability to integrate various modalities has seen rapid growth. Most previous multimodal models are trained on perfectly pai…
MissMAC-Bench: Building Solid Benchmark for Missing Modality Issue in Robust Multimodal Affective Computing
Ronghao Lin, Honghao Lu, Ruixing Wu +5
Current Multimodal Affective Computing (MAC) systems heavily rely on the completeness of multiple modalities to accurately understand human's affective state. However, in real-worl…
GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis
Manning Gao, Leheng Zhang, Shiqin Han +3
Most Multimodal Sentiment Analysis research has focused on point-wise regression. While straightforward, this approach is sensitive to label noise and neglects whether one sample i…
Towards Minimal Causal Representations for Human Multimodal Language Understanding
Menghua Jiang, Yuncheng Jiang, Haifeng Hu +1
Human Multimodal Language Understanding (MLU) aims to infer human intentions by integrating related cues from heterogeneous modalities. Existing works predominantly follow a ``lear…
Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis
Shiqin Han, Manning Gao, Menghua Jiang +3
Multimodal Large Language Models (MLLMs) have notably enhanced the performance of Multimodal Sentiment Analysis (MSA), yet their massive parameter scale leads to excessive resource…
E3RG: Building Explicit Emotion-driven Empathetic Response Generation System with Multimodal Large Language Model
Ronghao Lin, Shuai Shen, Weipeng Hu +5
Multimodal Empathetic Response Generation (MERG) is crucial for building emotionally intelligent human-computer interactions. Although large language models (LLMs) have improved te…
Multi-source Multimodal Progressive Domain Adaption for Audio-Visual Deception Detection
Ronghao Lin, Sijie Mai, Ying Zeng +3
This paper presents the winning approach for the 1st MultiModal Deception Detection (MMDD) Challenge at the 1st Workshop on Subtle Visual Computing (SVC). Aiming at the domain shif…
Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
Menghua Jiang, Yuxia Lin, Baoliang Chen +3
Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and visual data. However, existing m…
Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis
Sijie Mai, Yu Zhao, Ying Zeng +2
Multimodal sentiment analysis aims to effectively integrate information from various sources to infer sentiment, where in many cases there are no annotations for unimodal labels. T…
End-to-end Semantic-centric Video-based Multimodal Affective Computing
Ronghao Lin, Ying Zeng, Sijie Mai +1
In the pathway toward Artificial General Intelligence (AGI), understanding human's affection is essential to enhance machine's cognition abilities. For achieving more sensual human…
EE-TTS: Emphatic Expressive TTS with Linguistic Information
Yi Zhong, Chen Zhang, Xule Liu +4
While Current TTS systems perform well in synthesizing high-quality speech, producing highly expressive speech remains a challenge. Emphasis, as a critical factor in determining th…
Curriculum Learning Meets Weakly Supervised Modality Correlation Learning
Sijie Mai, Ya Sun, Haifeng Hu
In the field of multimodal sentiment analysis (MSA), a few studies have leveraged the inherent modality correlation information stored in samples for self-supervised learning. Howe…
Relation-dependent Contrastive Learning with Cluster Sampling for Inductive Relation Prediction
Jianfeng Wu, Sijie Mai, Haifeng Hu
Relation prediction is a task designed for knowledge graph completion which aims to predict missing relationships between entities. Recent subgraph-based models for inductive relat…
Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis
Ronghao Lin, Haifeng Hu
Multimodal representation learning is a challenging task in which previous work mostly focus on either uni-modality pre-training or cross-modality fusion. In fact, we regard modeli…
Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations
Sijie Mai, Ying Zeng, Haifeng Hu
Learning effective joint embedding for cross-modal data has always been a focus in the field of multimodal machine learning. We argue that during multimodal fusion, the generated m…
Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination
Rui Zhao, Jinming Song, Yufeng Yuan +5
We study the problem of training a Reinforcement Learning (RL) agent that is collaborative with humans without using any human data. Although such agents can be obtained through se…
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis
Ying Zeng, Sijie Mai, Haifeng Hu
Multimodal sentiment analysis (MSA) draws increasing attention with the availability of multimodal data. The boost in performance of MSA models is mainly hindered by two problems.…
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Sijie Mai, Ying Zeng, Shuangjia Zheng +1
The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis (MSA), most pr…
Subgraph-aware Few-Shot Inductive Link Prediction via Meta-Learning
Shuangjia Zheng, Sijie Mai, Ya Sun +2
Link prediction for knowledge graphs aims to predict missing connections between entities. Prevailing methods are limited to a transductive setting and hard to process unseen entit…