15 papers
Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis
Menghua Jiang, Haokai Gao, Xiangui Kang +2
Multimodal Sentiment Analysis (MSA) aims to infer human emotions from multiple modalities such as text, audio, and vision. In practice, inputs are often corrupted by noise and miss…
Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment
Baoliang Chen, Qing Lin, Sijie Mai
AI-generated image quality assessment (AIGIQA) requires jointly reasoning about perceptual fidelity and prompt alignment, two quality dimensions that are often treated as independe…
Uncovering Latent Depression Severity for Binary Depression Detection via Advantage-weighting Ranking
Manning Gao, Tingyi Liu, Leheng Zhang +3
Automatic depression detection using audio-visual data faces significant challenges, particularly in disentangling overlapping feature distributions and establishing robust decisio…
A Conflict-Aware Penalty and Statistical Loss Framework for Balancing Modalities and Enhancing Stability in Multimodal Sentiment Analysis
Jianheng Dai, Jiazhang Liang, Sijie Mai
Multimodal Sentiment Analysis (MSA) fuses text, acoustic, and visual streams to infer sentiment. Because pre-trained text encoders are far more expressive than their acoustic and v…
Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective
Sijie Mai, Shiqin Han
Multimodal affective computing aims to predict humans' sentiment, emotion, intention, and opinion using language, acoustic, and visual modalities. However, current models often lea…
C2F-Thinker: Coarse-to-Fine Reasoning with Hint-Guided Reinforcement Learning for Multimodal Sentiment Analysis
Miaosen Luo, Zhenhao Yang, Jieshen Long +3
Multimodal sentiment analysis aims to integrate textual, acoustic, and visual information for deep emotional understanding. Despite the progress of multimodal large language models…
CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusion
Sijie Mai, Shiqin Han
Modality gap significantly restricts the effectiveness of multimodal fusion. Previous methods often use techniques such as diffusion models and adversarial learning to reduce the m…
GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis
Manning Gao, Leheng Zhang, Shiqin Han +3
Most Multimodal Sentiment Analysis research has focused on point-wise regression. While straightforward, this approach is sensitive to label noise and neglects whether one sample i…
QASA: Quality-Aware Semantic Augmentation for Robust Multimodal Sentiment Analysis
Jiazhang Liang, Jianheng Dai, Miaosen Luo +2
Multimodal large language models have demonstrated strong ability in capturing semantic representations for multimodal sentiment analysis. Their capacity to learn stable and genera…
Beyond Cosine Similarity: Magnitude-Aware CLIP for No-Reference Image Quality Assessment
Zhicheng Liao, Dongxu Wu, Zhenshan Shi +5
Recent efforts have repurposed the Contrastive Language-Image Pre-training (CLIP) model for No-Reference Image Quality Assessment (NR-IQA) by measuring the cosine similarity betwee…
Towards Minimal Causal Representations for Human Multimodal Language Understanding
Menghua Jiang, Yuncheng Jiang, Haifeng Hu +1
Human Multimodal Language Understanding (MLU) aims to infer human intentions by integrating related cues from heterogeneous modalities. Existing works predominantly follow a ``lear…
Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis
Shiqin Han, Manning Gao, Menghua Jiang +3
Multimodal Large Language Models (MLLMs) have notably enhanced the performance of Multimodal Sentiment Analysis (MSA), yet their massive parameter scale leads to excessive resource…
Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Prompting
Miaosen Luo, Jiesen Long, Zequn Li +3
Multimodal Affective Computing (MAC) aims to recognize and interpret human emotions by integrating information from diverse modalities such as text, video, and audio. Recent advanc…
Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
Menghua Jiang, Yuxia Lin, Baoliang Chen +3
Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and visual data. However, existing m…
Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis
Miaosen Luo, Yuncheng Jiang, Sijie Mai
Multimodal Sentiment Analysis (MSA) faces two critical challenges: the lack of interpretability in the decision logic of multimodal fusion and modality imbalance caused by disparit…