9 papers
Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
Chao Hao, Jun Xu, Ji Du +6
Language-guided segmentation transcends the scope limitations of traditional semantic segmentation, enabling models to segment arbitrary target regions based on natural language in…
Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation
Zhixuan Shen, Jiawei Du, Ziyu Guo +5
Vision-Language Models (VLMs) have demonstrated exceptional general reasoning capabilities. However, their performance in embodied navigation remains hindered by a scarcity of alig…
Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
Xin Zhang, Qiqi Tao, Jiawei Du +2
Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models, enabling high-dimensional visual evidence to be integrated without…
Self-Calibrated Consistency can Fight Back for Adversarial Robustness in Vision-Language Models
Jiaxiang Liu, Jiawei Du, Xiao Liu +2
Pre-trained vision-language models (VLMs) such as CLIP have demonstrated strong zero-shot capabilities across diverse domains, yet remain highly vulnerable to adversarial perturbat…
Modest-Align: Data-Efficient Alignment for Vision-Language Models
Jiaxiang Liu, Yuan Wang, Jiawei Du +3
Cross-modal alignment aims to map heterogeneous modalities into a shared latent space, as exemplified by models like CLIP, which benefit from large-scale image-text pretraining for…
Beyond Modality Collapse: Representations Blending for Multimodal Dataset Distillation
Xin Zhang, Ziruo Zhang, Jiawei Du +2
Multimodal Dataset Distillation (MDD) seeks to condense large-scale image-text datasets into compact surrogates while retaining their effectiveness for cross-modal learning. Despit…