9 papers
SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
Chao Lei, Yanbei Jiang, Markus Hiller +4
Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions.…
Mind the Perspective: Let's Reason Recursively for Theory of Mind
Chao Lei, Guang Hu, Meng Yang +2
Theory of Mind (ToM) reasoning requires inferring agents' beliefs from partial and asymmetric observations, which remains an open challenge for LLMs. Existing prompting-based appro…
Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning
Yanbei Jiang, Amr Keleg, Ryandito Diandaru +4
While the real world is inherently stochastic, Large Language Models (LLMs) are predominantly evaluated on single-round inference against fixed ground truths. In this work, we shif…
Attention in Space: Functional Roles of VLM Heads for Spatial Reasoning
Xueqi Ma, Shuo Yang, Yanbei Jiang +6
Despite remarkable advances in large Vision-Language Models (VLMs), spatial reasoning remains a persistent challenge. In this work, we investigate how attention heads within VLMs c…
Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules
Yanbei Jiang, Xueqi Ma, Shu Liu +5
Despite excelling on multimodal benchmarks, vision-language models (VLMs) largely remain a black box. In this paper, we propose a novel interpretability framework to systematically…
Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning
Xueqi Ma, Jun Wang, Yanbei Jiang +3
Large language models (LLMs) have achieved state-of-the-art performance in a variety of tasks, but remain largely opaque in terms of their internal mechanisms. Understanding these…