12 papers
JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making
Zhongwei Xie, Xingsu Li, Jingqiao Yu +8
Repeated event forecasting requires extracting predictive signals from noisy evidence and evaluating their value across successive decisions. We introduce JockeyBench, a benchmark…
Jev-ToM: Evaluating Theory of Mind in Typed Decision Models
Kawai Chung, Chunkit Chan, Yuxuan Liu +9
Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative to generative large language…
Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Trees
Wenbin Hu, Huihao Jing, Haochen Shi +9
Agent harnesses require a low-latency guardrail model together with customizable audit policies. However, deploying an auto-regressive guardrail model remains challenging: such mod…
RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving
Huihao Jing, Wenbin Hu, Shaojin Chen +6
Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in the current worker round. Sc…
Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization
Huihao Jing, Shaojin Chen, Wenbin Hu +5
As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central safety problem. Guardrails are…
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
Yuxuan Liu, Zhaochen Su, Hangyu Guo +11
Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requir…
SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown
Yuxuan Liu, Haoran Li, Yuhao Zhang +10
Executable Agent Skills combine natural-language instructions and scripts into reusable packages for LLM agents, and revising them requires fixing errors without breaking correct b…
MultivationBench: A Benchmark for Multimodal Sequential Motivation Reasoning
Kawai Chung, Chunkit Chan, Yauwai Yim +12
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning…
Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
Qing Zong, Jiayu Liu, Junhao Shen +9
Agentic systems are increasingly expected to improve after deployment, yet single-entity self-evolution is often bounded by a static learning context, such as fixed tasks and feedb…
Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs
Hongyu Luo, He Wang, Huihao Jing +6
Current evaluations do not isolate whether text-only language models can originate visual concepts before image generation. Fluent visual prose can hide visual-plan failures: an an…
Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds
Yuxuan Liu, Zhaochen Su, Yuhao Zhang +9
Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model. Yet it remains unclear when…
RLPF: Reinforcement Learning from Performance Feedback for Code Generation
Huihao Jing, Haozhe Cui, Wenbin Hu +9
Code models are increasingly trained with execution feedback, but most training signals still stop at correctness. This leaves an important gap for systems code: two programs can p…