4 papers
Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents
Kaiming Liu, Yuxuan Liu, Jinrui Ju +4
Large language model agents can follow complex human instructions to solve real-world tasks. In practice, however, initial instructions are often incomplete. In our analysis of 100…
BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions
Minyuan Ruan, Ziyue Wang, Kaiming Liu +3
Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instru…
Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction
Dairu Liu, Ziyue Wang, Minyuan Ruan +4
Images usually convey richer detail than text, but often include redundant information, which potentially downgrades multimodal reasoning performance. When faced with lengthy or co…
EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability
Ziyue Wang, Yurui Dong, Fuwen Luo +5
The rapid advancing of Multimodal Large Language Models (MLLMs) has spurred interest in complex multimodal reasoning tasks in the real-world and virtual environment, which require…