7 papers · 1 filter
LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory
Kun-Yang Yu, Yingzhe Li, Hongyu Xu +8
Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal La…
Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models
Yang Chen, Zhan Zhuang, Yanbin Wei +3
While adversarial prompt tuning can enhance robustness of vision-language models efficiently, we find that existing methods aggravate robust generalization overfitting on seen clas…
LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models
Shi-Yu Tian, Zhi Zhou, Kun-Yang Yu +5
Spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world. However, multimodal large language models (MLLMs) frequently…
See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
Yongchang Zhang, Oliver Ma, Tianyi Liu +2
Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal…
DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs
Yanbin Wei, Jiangyue Yan, Chun Kang +4
Vision-Language Models (VLMs) have emerged as versatile solutions for zero-shot question answering (QA) across various domains. However, enabling VLMs to effectively comprehend str…
Alternating Training-based Label Smoothing Enhances Prompt Generalization
Yang Chen, Yanbin Wei, Ke Jin +3
Recent advances in pre-trained vision-language models have demonstrated remarkable zero-shot generalization capabilities. To further enhance these models' adaptability to various d…
Open Your Eyes: Vision Enhances Message Passing Neural Networks in Link Prediction
Yanbin Wei, Xuehao Wang, Zhan Zhuang +5
Message-passing graph neural networks (MPNNs) and structural features (SFs) are cornerstones for the link prediction task. However, as a common and intuitive mode of understanding,…