6 papers · 1 filter
JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making
Zhongwei Xie, Xingsu Li, Jingqiao Yu +8
Repeated event forecasting requires extracting predictive signals from noisy evidence and evaluating their value across successive decisions. We introduce JockeyBench, a benchmark…
RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving
Huihao Jing, Wenbin Hu, Shaojin Chen +6
Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in the current worker round. Sc…
Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization
Huihao Jing, Shaojin Chen, Wenbin Hu +5
As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central safety problem. Guardrails are…
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
Yuxuan Liu, Zhaochen Su, Hangyu Guo +11
Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requir…
SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown
Yuxuan Liu, Haoran Li, Yuhao Zhang +10
Executable Agent Skills combine natural-language instructions and scripts into reusable packages for LLM agents, and revising them requires fixing errors without breaking correct b…
MultivationBench: A Benchmark for Multimodal Sequential Motivation Reasoning
Kawai Chung, Chunkit Chan, Yauwai Yim +12
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning…