10 papers
Visual Attention Faithfulness in Vision-Language Models is Heterogeneous
Xurui Song, Weishi Wang, Zhongqi Yue +5
Whether attention weights faithfully reflect model reasoning has been actively debated in NLP, yet this question remains largely unexplored for the visual modality in Vision-Langua…
SCOPE: Evolving Symbolic World for Planning in Open-Ended Environments
Yundaichuan Zhan, Minghe Gao, Zhongqi Yue +7
Recent works have explored integrating Vision-Language Models (VLMs) with classical planners that rely on symbolic representations of planning problems to generate long-horizon pla…
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
Jianbo Lin, Xiaomin Yu, Yi Xin +7
Large language model-based agents make mistakes, yet critique can often guide the same model toward correct behavior. However, when critique is removed, the model may fail again on…
When are radiology reports useful for training medical image classifiers?
Herman Bergström, Zhongqi Yue, Fredrik D. Johansson
Medical images used to train machine learning models are often accompanied by radiology reports containing rich expert annotations. However, relying on these reports as inputs for…
Expanding the Action Space of LLMs to Reason Beyond Language
Zhongqi Yue, Weishi Wang, Yundaichuan Zhan +3
Large Language Models (LLMs) are powerful reasoners in natural language, but their actions are typically confined to outputting vocabulary tokens. As a result, interactions with ex…
Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning
Bohan Wang, Zhongqi Yue, Fengda Zhang +15
We completely discard the conventional spatial prior in image representation and introduce a novel discrete visual tokenizer: Self-consistency Tokenizer (Selftok). At its design co…