16 papers
Illusion of Alignment: Detecting Hidden Disagreement in Collaborative Dialogue
Kaiming Liu, Fuwen Luo, Ziyue Wang +6
Collaborative dialogue can end with apparent agreement while participants still differ on goals, assumptions, or execution plans, creating an \textbf{illusion of alignment (IoA)}.…
Evaluating Time Awareness and Cross-modal Active Perception of Large Models via 4D Escape Room Task
Yurui Dong, Ziyue Wang, Shuyun Lu +5
Multimodal Large Language Models (MLLMs) have recently made rapid progress toward unified Omni models that integrate vision, language, and audio. However, existing environments lar…
BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions
Minyuan Ruan, Ziyue Wang, Kaiming Liu +3
Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instru…
Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors
Fuwen Luo, Zihao Wan, Ziyue Wang +6
Hieroglyphs, as logographic writing systems, encode rich semantic and cultural information within their internal structural composition. Yet, current advanced Large Language Models…
Bridging Vision, Language, and Mathematics: Pictographic Character Reconstruction with Bézier Curves
Zihao Wan, Pau Tong Lin Xu, Fuwen Luo +3
While Vision-language Models (VLMs) have demonstrated strong semantic capabilities, their ability to interpret the underlying geometric structure of visual information is less expl…
Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning
Yunghwei Lai, Kaiming Liu, Ziyue Wang +2
The professionalism of a human doctor in outpatient service depends on two core abilities: the ability to make accurate medical decisions and the medical consultation skill to cond…
Patient-Zero: Scaling Synthetic Patient Agents to Real-World Distributions without Real Patient Data
Yunghwei Lai, Ziyue Wang, Weizhi Ma +1
Synthetic data generation with Large Language Models (LLMs) has emerged as a promising solution in the medical domain to mitigate data scarcity and privacy constraints. However, ex…
MUCAR: Benchmarking Multilingual Cross-Modal Ambiguity Resolution for Multimodal Large Language Models
Xiaolong Wang, Zhaolu Kang, Wangyuxuan Zhai +8
Multimodal Large Language Models (MLLMs) have demonstrated significant advances across numerous vision-language tasks. MLLMs have shown promising capability in aligning visual and…
Agent-Environment Alignment via Automated Interface Generation
Kaiming Liu, Xuanyu Lei, Ziyue Wang +2
Large language model (LLM) agents have shown impressive reasoning capabilities in interactive decision-making tasks. These agents interact with environment through intermediate int…
MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding
Fuwen Luo, Shengfeng Lou, Chi Chen +9
Video temporal understanding is crucial for multimodal large language models (MLLMs) to reason over events in videos. Despite recent advances in general video understanding, curren…
Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction
Dairu Liu, Ziyue Wang, Minyuan Ruan +4
Images usually convey richer detail than text, but often include redundant information, which potentially downgrades multimodal reasoning performance. When faced with lengthy or co…
CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models
Yiqi Zhu, Ziyue Wang, Can Zhang +2
Vision-Language Models (VLMs) have recently witnessed significant progress in visual comprehension. As the permitting length of image context grows, VLMs can now comprehend a broad…
EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability
Ziyue Wang, Yurui Dong, Fuwen Luo +5
The rapid advancing of Multimodal Large Language Models (MLLMs) has spurred interest in complex multimodal reasoning tasks in the real-world and virtual environment, which require…
DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms
Xiaojun Bi, Shuo Li, Junyao Xing +7
Dongba pictographic is the only pictographic script still in use in the world. Its pictorial ideographic features carry rich cultural and contextual information. However, due to th…
Perspective Transition of Large Language Models for Solving Subjective Tasks
Xiaolong Wang, Yuanchi Zhang, Ziyue Wang +5
Large language models (LLMs) have revolutionized the field of natural language processing, enabling remarkable progress in various tasks. Different from objective tasks such as com…
ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models
Ziyue Wang, Chi Chen, Fuwen Luo +6
Active perception, a crucial human capability, involves setting a goal based on the current understanding of the environment and performing actions to achieve that goal. Despite si…