1 citations · 1 across the 14 of their papers we have counts for
12 papers · 1 filter
Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Zhefan Rao, Meng Chu, Xuan Zhang +80
World Models (WMs) can support Multimodal Agents (MAs) through simulated experience, while Agent trajectories can inform World Model refinement. Evaluating this interaction require…
InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems
Enxin Song, Suhao Yu, Yifei Xu +7
A video assistant must speak when its instruction warrants a response and stay silent otherwise. We introduce a benchmark that evaluates this decision for the complete system of mo…
Video-Index: A Curated Meta-Benchmark for Video Understanding
Enxin Song, Yinuo Xu, Shusheng Yang +2
A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. We introduce the attack pyram…
EasyPPO: Stabilizing the Critic Is Key
Xuanyi Zhou, Qiuyang Mang, Huanzhi Mao +7
A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns…
An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond
Wenbo Zhang, Kaixuan Wang, Yutao Ouyang +9
Embodied AI systems are often organized into System 1 and System 2. System 1 is typically a pretrained policy that generates actions at high frequency, whereas System 2 is often in…
When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
Kaiyuan Liu, Qiuyang Mang, Bo Peng +6
Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to stop. This test-time strategy…
MatrAIx: Simulating the World with 8.3 Billion Persona Agents
Xiaomin Li, Yuexing Hao, Jianheng Hou +90
Human evaluation of AI systems and digital products is costly, slow, and difficult to scale. Offline evaluations are more scalable but often abstract away human diversity and inter…
Learning What to Remember: Test-Time Training via Context Distillation
Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4
Effective long-context modeling is not merely about retaining more of the past, but about preserving the information that may prove relevant later. Test-time training (TTT) is an a…
RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
Tianxing Chen, Yue Chen, Zixuan Li +41
Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-hor…
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
Runyuan He, Qiuyang Mang, Shang Zhou +14
Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implement…
UEval: A Benchmark for Unified Multimodal Generation
Bo Li, Yida Yin, Wenhao Chai +2
We introduce UEval, a benchmark to evaluate unified models, i.e., models capable of generating both images and text. UEval comprises 1,000 expert-curated questions that require bot…
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini +82
AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not…