10 papers
SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
Chao Lei, Yanbei Jiang, Markus Hiller +4
Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions.…
Mind the Perspective: Let's Reason Recursively for Theory of Mind
Chao Lei, Guang Hu, Meng Yang +2
Theory of Mind (ToM) reasoning requires inferring agents' beliefs from partial and asymmetric observations, which remains an open challenge for LLMs. Existing prompting-based appro…
Planning as Goal Recognition: Deriving Heuristics from Intention Models -- Extended Version
Giacomo Rosa, Jean Honorio, Nir Lipovetzky +1
Classical planning aims to find a sequence of actions, a plan, that maps a starting state into one of the goal states. If a trajectory appears to be leading to the goal, should we…
Generative AI-assisted Participatory Modeling in Socio-Environmental Planning under Deep Uncertainty
Zhihao Pei, Nir Lipovetzky, Angela M. Rojas-Arevalo +2
Socio-environmental planning under deep uncertainty requires researchers to identify and conceptualize problems before exploring policies and deploying plans. In practice and model…
The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards
Sukai Huang, Shu-Wei Liu, Nir Lipovetzky +1
While Vision-Language Models (VLMs) are increasingly used to generate reward signals for training embodied agents to follow instructions, our research reveals that agents guided by…
Where Common Knowledge Cannot Be Formed, Common Belief Can -- Planning with Multi-Agent Belief Using Group Justified Perspectives
Guang Hu, Tim Miller, Nir Lipovetzky
Epistemic planning is the sub-field of AI planning that focuses on changing knowledge and belief. It is important in both multi-agent domains where agents need to have knowledge/be…