16 papers
Calibrated Multimodal Representation Learning with Missing Modalities
Xiaohao Liu, Xiaobo Xia, Jiaheng Wei +4
Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space. Recent research generalizes traditional cross-modal alignment to pro…
AUHead: Realistic Emotional Talking Head Generation via Action Units Control
Jiayi Lyu, Leigang Qu, Wenjing Zhang +6
Realistic talking-head video generation is critical for virtual avatars, film production, and interactive systems. Current methods struggle with nuanced emotional expressions due t…
Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models
Enyi Shi, Pengyang Shao, Yanxin Zhang +5
The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current benchmarks typically isolate the…
Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization
Wenhao Yang, Yu Xia, Jinlong Huang +10
Recent advancements in Multimodal Large Language Models (MLLMs) have incentivized models to ``think with images'' by actively invoking visual tools during multi-turn reasoning. The…
Principled Multimodal Representation Learning
Xiaohao Liu, Xiaobo Xia, See-Kiong Ng +1
Multimodal representation learning seeks to create a unified representation space by integrating diverse data modalities to improve multimodal understanding. Traditional methods of…
Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models
Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai +8
Due to constraints on privacy, cost, and latency, on-premise deployment of small models is increasingly common. However, most practical pipelines stop at supervised fine-tuning (SF…