13 papers
LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory
Kun-Yang Yu, Yingzhe Li, Hongyu Xu +8
Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal La…
STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation
Yang Chen, Zhenyu Huang, Wenbo Fu +4
Multimodal lifelong navigation requires an agent to autonomously explore unseen environments while sequentially completing navigation tasks specified by object categories, language…
Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models
Yang Chen, Zhan Zhuang, Yanbin Wei +3
While adversarial prompt tuning can enhance robustness of vision-language models efficiently, we find that existing methods aggravate robust generalization overfitting on seen clas…
Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning
Shi-Yu Tian, Zhuo-Xia Wang, Xuan-Yi Zhu +6
Large vision-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine-grained spatial tasks that demand both precise spatial per…
VizRAG: Enhancing Retrieval-Augmented Generation with Hypergraph Visualization
Yanbin Wei, Yang Chen, Renling Gan +7
Hypergraph-based RAG systems surpass traditional graph-based approaches by organizing complex n-ary atomic facts among entities, rather than relying solely on binary relationships.…
HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
Yanbin Wei, Chun Kang, Siwei Li +11
Large Vision-Language Models (LVLMs) consistently require new arenas to guide their expanding boundaries, yet their capabilities with hypergraphs remain unexplored. In the real wor…
LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models
Shi-Yu Tian, Zhi Zhou, Kun-Yang Yu +5
Spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world. However, multimodal large language models (MLLMs) frequently…
See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
Yongchang Zhang, Oliver Ma, Tianyi Liu +2
Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal…
DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs
Yanbin Wei, Jiangyue Yan, Chun Kang +4
Vision-Language Models (VLMs) have emerged as versatile solutions for zero-shot question answering (QA) across various domains. However, enabling VLMs to effectively comprehend str…
Alternating Training-based Label Smoothing Enhances Prompt Generalization
Yang Chen, Yanbin Wei, Ke Jin +3
Recent advances in pre-trained vision-language models have demonstrated remarkable zero-shot generalization capabilities. To further enhance these models' adaptability to various d…
Harnessing Adaptive Topology Representations for Zero-Shot Graph Question Answering
Yanbin Wei, Jiangyue Yan, Chun Kang +4
Large Multimodal Models (LMMs) have shown generalized zero-shot capabilities in diverse domain question-answering (QA) tasks, including graph QA that involves complex graph topolog…
DynamicMind: A Tri-Mode Thinking System for Large Language Models
Wei Li, Yanbin Wei, Qiushi Huang +4
Modern large language models (LLMs) often struggle to dynamically adapt their reasoning depth to varying task complexities, leading to suboptimal performance or inefficient resourc…
Open Your Eyes: Vision Enhances Message Passing Neural Networks in Link Prediction
Yanbin Wei, Xuehao Wang, Zhan Zhuang +5
Message-passing graph neural networks (MPNNs) and structural features (SFs) are cornerstones for the link prediction task. However, as a common and intuitive mode of understanding,…