16 papers
Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents
Kaiming Liu, Yuxuan Liu, Jinrui Ju +4
Large language model agents can follow complex human instructions to solve real-world tasks. In practice, however, initial instructions are often incomplete. In our analysis of 100…
Illusion of Alignment: Detecting Hidden Disagreement in Collaborative Dialogue
Kaiming Liu, Fuwen Luo, Ziyue Wang +6
Collaborative dialogue can end with apparent agreement while participants still differ on goals, assumptions, or execution plans, creating an \textbf{illusion of alignment (IoA)}.…
Bridging the Agent-World Gap: Text World Models for LLM-based Agents
Yixia Li, Hongru Wang, Peng Lai +13
Large language model (LLM)-based agents are increasingly used in interactive textual environments, from web navigation and code editing to tool use and long-horizon dialogue. Yet m…
Evaluating Time Awareness and Cross-modal Active Perception of Large Models via 4D Escape Room Task
Yurui Dong, Ziyue Wang, Shuyun Lu +5
Multimodal Large Language Models (MLLMs) have recently made rapid progress toward unified Omni models that integrate vision, language, and audio. However, existing environments lar…
TacMamba: A Tactile History Compression Adapter Bridging Fast Reflexes and Slow VLA Reasoning
Zhenan Wang, Yanzhe Wang, Meixuan Ren +8
In visually ambiguous manipulation such as detecting button click tactile feedback is often the sole source of ground truth. However, fusing tactile data poses a significant challe…
BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions
Minyuan Ruan, Ziyue Wang, Kaiming Liu +3
Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instru…
Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors
Fuwen Luo, Zihao Wan, Ziyue Wang +6
Hieroglyphs, as logographic writing systems, encode rich semantic and cultural information within their internal structural composition. Yet, current advanced Large Language Models…
Bridging Vision, Language, and Mathematics: Pictographic Character Reconstruction with Bézier Curves
Zihao Wan, Pau Tong Lin Xu, Fuwen Luo +3
While Vision-language Models (VLMs) have demonstrated strong semantic capabilities, their ability to interpret the underlying geometric structure of visual information is less expl…
MUCAR: Benchmarking Multilingual Cross-Modal Ambiguity Resolution for Multimodal Large Language Models
Xiaolong Wang, Zhaolu Kang, Wangyuxuan Zhai +8
Multimodal Large Language Models (MLLMs) have demonstrated significant advances across numerous vision-language tasks. MLLMs have shown promising capability in aligning visual and…
Agent-Environment Alignment via Automated Interface Generation
Kaiming Liu, Xuanyu Lei, Ziyue Wang +2
Large language model (LLM) agents have shown impressive reasoning capabilities in interactive decision-making tasks. These agents interact with environment through intermediate int…
Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration
Zijun Liu, Zhennan Wan, Peng Li +3
With the rapid advancement of post-training techniques for reasoning and information seeking, large language models (LLMs) can incorporate a large quantity of retrieved knowledge t…
MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding
Fuwen Luo, Shengfeng Lou, Chi Chen +9
Video temporal understanding is crucial for multimodal large language models (MLLMs) to reason over events in videos. Despite recent advances in general video understanding, curren…
Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction
Dairu Liu, Ziyue Wang, Minyuan Ruan +4
Images usually convey richer detail than text, but often include redundant information, which potentially downgrades multimodal reasoning performance. When faced with lengthy or co…
EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability
Ziyue Wang, Yurui Dong, Fuwen Luo +5
The rapid advancing of Multimodal Large Language Models (MLLMs) has spurred interest in complex multimodal reasoning tasks in the real-world and virtual environment, which require…
DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms
Xiaojun Bi, Shuo Li, Junyao Xing +7
Dongba pictographic is the only pictographic script still in use in the world. Its pictorial ideographic features carry rich cultural and contextual information. However, due to th…
StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
Junming Lin, Zheng Fang, Chi Chen +5
The rapid development of Multimodal Large Language Models (MLLMs) has expanded their capabilities from image comprehension to video understanding. However, most of these MLLMs focu…