3 citations · 18 across the 48 of their papers we have counts for
11 papers · 1 filter
JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making
Zhongwei Xie, Xingsu Li, Jingqiao Yu +8
Repeated event forecasting requires extracting predictive signals from noisy evidence and evaluating their value across successive decisions. We introduce JockeyBench, a benchmark…
Beyond the Verdict: Agentic Code Judging with Verifiable Feedback
Zheye Deng, Sha Li, Changlong Yu +4
Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To guide selection and revision…
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
Yuxuan Liu, Zhaochen Su, Hangyu Guo +11
Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requir…
AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
Yunxiang Mo, Tianshi Zheng, Yisen Gao +7
Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Existing evaluations largely ass…
MultivationBench: A Benchmark for Multimodal Sequential Motivation Reasoning
Kawai Chung, Chunkit Chan, Yauwai Yim +12
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning…
The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas
Baixuan Xu, Tianshi Zheng, Zhaowei Wang +4
Enabling LLMs to effectively operate long-horizon task which requires long-term planning and multiple interactions is essential for open-world autonomy. Conventional methods adopt…
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping
Wei Fan, Wenlin Yao, Zheng Li +6
Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that requir…
NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
Tianshi Zheng, Kelvin Kiu-Wai Tam, Newt Hue-Nam K. Nguyen +10
Large language models are emerging as powerful tools for scientific law discovery, a foundational challenge in AI-driven science. However, existing benchmarks for this task suffer…
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
Rui Wang, Qihan Lin, Jiayu Liu +7
Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human…
Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations
Chunyang Li, Weiqi Wang, Tianshi Zheng +1
Inductive reasoning, a cornerstone of human cognition, enables generalization from limited data but hasn't yet been fully achieved by large language models (LLMs). While modern LLM…
TILFA: A Unified Framework for Text, Image, and Layout Fusion in Argument Mining
Qing Zong, Zhaowei Wang, Baixuan Xu +6
A main goal of Argument Mining (AM) is to analyze an author's stance. Unlike previous AM datasets focusing only on text, the shared task at the 10th Workshop on Argument Mining int…