7 papers · 1 filter
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +306
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation
Shengtao Zheng, Kai Li, Weichen Zhang +5
End-to-end Vision-Language-Action (VLA) models have shown promise in UAV navigation. However, existing approaches typically rely on historical observations to directly predict acti…
How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
Baining Zhao, Ziyou Wang, Jianjie Fang +8
Large multimodal models (LMMs) show strong visual-linguistic reasoning but their capacity for spatial decision-making and action remains unclear. In this work, we investigate wheth…
IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks
Xiaoya Lu, Zeren Chen, Xuhao Hu +5
Flawed planning from VLM-driven embodied agents poses significant safety hazards, hindering their deployment in real-world household tasks. However, existing static, non-interactiv…
MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
Weichen Zhang, Yiyou Sun, Pohao Huang +3
Hallucinations pose critical risks for large language model (LLM)-based agents, often manifesting as hallucinative actions resulting from fabricated or misinterpreted information w…
Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report
Shanghai AI Lab, :, Xiaoyang Chen +35
To understand and identify the unprecedented risks posed by rapidly advancing artificial intelligence (AI) models, this report presents a comprehensive assessment of their frontier…