13 papers
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Xiangyi Li, Yimin Liu, Wenbo Chen +75
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to m…
AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor
Shu Yang, Jingyu Hu, Tong Li +3
We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes. Au…
AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
Haipeng Luo, Huawen Feng, Qingfeng Sun +6
Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in reasoning tasks with long cot. However, they remain computationally inefficient and strug…
Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models
Xilin Gong, Shu Yang, Zehua Cao +2
Large Language Models (LLMs) have demonstrated strong capabilities for hidden representation interpretation through Patchscopes, a framework that uses LLMs themselves to generate h…
Hearing is Believing? Evaluating and Analyzing Audio Language Model Sycophancy with SYAUDIO
Junchi Yao, Lokranjan Lakshmikanthan, Annie Zhao +5
Audio Language Models (ALMs) have recently shown strong capabilities in unified reasoning over speech, sound, and natural language; yet they inherit behavioral issues observed in L…
Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning
Lukas Twist, Shu Yang, Hanqi Yan +4
Large Language Models (LLMs) increasingly exhibit strong reasoning abilities, often attributed to their capacity to generate chain-of-thought-style intermediate reasoning. Recent w…