6 papers · 1 filter
REALHOP: Rethinking Multi-Hop Reasoning Evaluation via Behavioral Auditing
Jiawen Tao, Xiaokun Yuan, Yaoming Li +4
Complex questions often require multi-hop reasoning that connects facts distributed across sources or distant regions of a long context through intermediate steps. Benchmarks commo…
ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds
Ming Zhang, Zhenghao Xiang, Peizhong Gao +17
Scientific discovery begins where known problems end. There, AI systems must engage in exploration: framing hypotheses, designing experiments, and iterating on the results. However…
Breaking the Environment Wall: A Unified Framework for Preparing and Evolving Agent-Native Environments
Yukai Wu, Yuanjing Yang, Le Zhou +8
Many real-world tasks (e.g., office workflows, scientific experimentation) require LLM agents to interact repeatedly with their environments for context-dependent operations. Howev…
Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
Jiawen Tao, Miao Peng, Yaoming Li +7
Synthetic textbook data has improved language model pre-training, but prior work largely treats the benefit as a property of generated content or local rewriting style. We study a…
E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios
Weihuang Zheng, Tianyuan Zou, Eileen Ye +5
Large Language Models (LLMs) are increasingly deployed as agents that interact with stateful environments over multiple steps: gathering hidden information, composing tool calls, a…
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
Jiaqi Shao, Hanck Chen, Wei Zhang +2
Agent benchmarks increasingly evaluate repository editing, web research, terminal use, and long-horizon interaction. Their scores support capability claims only when the evaluation…