3 citations · 18 across the 48 of their papers we have counts for
54 papers
JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making
Zhongwei Xie, Xingsu Li, Jingqiao Yu +8
Repeated event forecasting requires extracting predictive signals from noisy evidence and evaluating their value across successive decisions. We introduce JockeyBench, a benchmark…
Beyond the Verdict: Agentic Code Judging with Verifiable Feedback
Zheye Deng, Sha Li, Changlong Yu +4
Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To guide selection and revision…
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
Yuxuan Liu, Zhaochen Su, Hangyu Guo +11
Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requir…
ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe
Weihan Li, Tianshi Zheng, Yangqiu Song +2
Scientific discovery often begins when scattered clues call for a new way of describing the world. Such abductive exploration can require constructing the representation in which a…
AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
Yunxiang Mo, Tianshi Zheng, Yisen Gao +7
Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Existing evaluations largely ass…
MultivationBench: A Benchmark for Multimodal Sequential Motivation Reasoning
Kawai Chung, Chunkit Chan, Yauwai Yim +12
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning…
Finding the Right Evidence: Factor-Guided Coarse-to-Fine Reasoning for Long Videos
Baixuan Xu, Yinyui Xu, Tianshi Zheng +9
While LVLMs rapidly improve, long-video question answering still remains challenging: relevant evidence is sparse, and question-relevant context often fails to provide cues that di…
Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
Qing Zong, Jiayu Liu, Junhao Shen +9
Agentic systems are increasingly expected to improve after deployment, yet single-entity self-evolution is often bounded by a static learning context, such as fixed tasks and feedb…
Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs
Hongyu Luo, He Wang, Huihao Jing +6
Current evaluations do not isolate whether text-only language models can originate visual concepts before image generation. Fluent visual prose can hide visual-plan failures: an an…
EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World
Qing Zong, Yue Guo, Mengxin Yang +2
This paper introduces EvolvingWorld, a framework and benchmark for character and world co-evolution in interactive literary worlds. Existing systems either treat interactive litera…
SciLens: Multi-modal Scientific Claim Verification with Agentic Entailment and Grounding
Yueming Wang, Tianshi Zheng, Jiaxin Bai +3
Scientific discovery increasingly relies on automated systems that generate hypotheses, inspect multimodal evidence, and validate claims at scale. Yet scientific claim verification…
SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents
Qiao Xiao, Haochen Shi, Yisen Gao +9
Large language model (LLM) agents increasingly rely on agent harnesses that manage context, tools, and multi-turn execution, making tools a central interface for acting in realisti…
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Weiqi Wang, Xin Liu, Binxuan Huang +13
RLVR has become a standard recipe for training LLMs on reasoning tasks with verifiable outcomes, but when rollout generation dominates the cost, efficiency hinges on which prompts…
CritiCal: Can Critique Help LLM Uncertainty or Confidence Calibration?
Qing Zong, Jiayu Liu, Tianshi Zheng +7
Accurate confidence calibration in Large Language Models (LLMs) is critical for safe use in high-stakes domains, where clear verbalized confidence enhances user trust. Traditional…
The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas
Baixuan Xu, Tianshi Zheng, Zhaowei Wang +4
Enabling LLMs to effectively operate long-horizon task which requires long-term planning and multiple interactions is essential for open-world autonomy. Conventional methods adopt…
WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality
Chunyang Li, Yilun Zheng, Xinting Huang +5
The paradigm of LLM-as-a-judge is emerging as a scalable and efficient alternative to human evaluation, demonstrating strong performance on well-defined tasks. However, its reliabi…
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping
Wei Fan, Wenlin Yao, Zheng Li +6
Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that requir…
NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
Tianshi Zheng, Kelvin Kiu-Wai Tam, Newt Hue-Nam K. Nguyen +10
Large language models are emerging as powerful tools for scientific law discovery, a foundational challenge in AI-driven science. However, existing benchmarks for this task suffer…
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
Rui Wang, Qihan Lin, Jiayu Liu +7
Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human…
SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding
Yuqi Yang, Weiqi Wang, Baixuan Xu +13
Session history is a common way of recording user interacting behaviors throughout a browsing activity with multiple products. For example, if an user clicks a product webpage and…
INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling
Haochen Shi, Tianshi Zheng, Weiqi Wang +6
Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs, aiming to select the best-performing LLMs tailored to the domains of user quer…
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association
Weiqi Wang, Limeng Cui, Xin Liu +14
Goal-oriented script planning, or the ability to devise coherent sequences of actions toward specific goals, is commonly employed by humans to plan for typical activities. In e-com…
Towards Multi-Agent Reasoning Systems for Collaborative Expertise Delegation: An Exploratory Design Study
Baixuan Xu, Chunyang Li, Weiqi Wang +6
Designing effective collaboration structure for multi-agent LLM systems to enhance collective reasoning is crucial yet remains under-explored. In this paper, we systematically inve…
Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?
Jiayu Liu, Qing Zong, Weiqi Wang +1
As large language models (LLMs) are increasingly used in high-stakes domains, accurately assessing their confidence is crucial. Humans typically express confidence through epistemi…