16 papers
Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization
Wenhao Yang, Yu Xia, Jinlong Huang +10
Recent advancements in Multimodal Large Language Models (MLLMs) have incentivized models to ``think with images'' by actively invoking visual tools during multi-turn reasoning. The…
AUHead: Realistic Emotional Talking Head Generation via Action Units Control
Jiayi Lyu, Leigang Qu, Wenjing Zhang +6
Realistic talking-head video generation is critical for virtual avatars, film production, and interactive systems. Current methods struggle with nuanced emotional expressions due t…
Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models
Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai +8
Due to constraints on privacy, cost, and latency, on-premise deployment of small models is increasingly common. However, most practical pipelines stop at supervised fine-tuning (SF…
Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models
Enyi Shi, Pengyang Shao, Yanxin Zhang +5
The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current benchmarks typically isolate the…
ITS3D: Inference-Time Scaling for Text-Guided 3D Diffusion Models
Zhenglin Zhou, Fan Ma, Xiaobo Xia +3
We explore inference-time scaling in text-guided 3D diffusion models to enhance generative quality without additional training. To this end, we introduce ITS3D, a framework that fo…
AnchorFlow: Training-Free 3D Editing via Latent Anchor-Aligned Flows
Zhenglin Zhou, Fan Ma, Chengzhuo Gui +4
Training-free 3D editing aims to modify 3D shapes based on human instructions without model finetuning. It plays a crucial role in 3D content creation. However, existing approaches…
Calibrated Multimodal Representation Learning with Missing Modalities
Xiaohao Liu, Xiaobo Xia, Jiaheng Wei +4
Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space. Recent research generalizes traditional cross-modal alignment to pro…
UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation
Jiyu Guo, Shuo Yang, Yiming Huang +6
Data augmentation using generative models has emerged as a powerful paradigm for enhancing performance in computer vision tasks. However, most existing augmentation approaches prim…
OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models
Hao Zheng, Zirui Pang, Ling li +5
Advances in Multimodal Large Language Models (MLLMs) intensify concerns about data privacy, making Machine Unlearning (MU), the selective removal of learned information, a critical…
Logic Unseen: Revealing the Logical Blindspots of Vision-Language Models
Yuchen Zhou, Jiayu Tang, Shuo Yang +6
Vision-Language Models (VLMs), exemplified by CLIP, have emerged as foundational for multimodal intelligence. However, their capacity for logical understanding remains significantl…
Principled Multimodal Representation Learning
Xiaohao Liu, Xiaobo Xia, See-Kiong Ng +1
Multimodal representation learning seeks to create a unified representation space by integrating diverse data modalities to improve multimodal understanding. Traditional methods of…
Where, What, Why: Towards Explainable Driver Attention Prediction
Yuchen Zhou, Jiayu Tang, Xiaoyan Xiao +6
Modeling task-driven attention in driving is a fundamental challenge for both autonomous vehicles and cognitive science. Existing methods primarily predict where drivers look by ge…
L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models
Xiaohao Liu, Xiaobo Xia, Weixiang Zhao +6
Large language models (LLMs) have achieved notable progress. Despite their success, next-token prediction (NTP), the dominant method for LLM training and inference, is constrained…
Continual Multimodal Contrastive Learning
Xiaohao Liu, Xiaobo Xia, See-Kiong Ng +1
Multimodal Contrastive Learning (MCL) advances in aligning different modalities and generating multimodal representations in a joint space. By leveraging contrastive learning acros…
DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
Zhenglin Zhou, Xiaobo Xia, Fan Ma +3
Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle…
Towards Modality Generalization: A Benchmark and Prospective Analysis
Xiaohao Liu, Xiaobo Xia, Zhuo Huang +2
Multi-modal learning has achieved remarkable success by integrating information from various modalities, achieving superior performance in tasks like recognition and retrieval comp…