5 papers
Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds
Yefan Tao, Gerald Friedland, Madhusudhanan Chandrasekaran +1
Reflection, the ability to revisit and revise prior reasoning, is central to how humans improve their answers. Large language models (LLMs) are increasingly prompted to "reflect,"…
Lean4Agent: Formal Modeling and Verification for Agent Workflow and Trajectory
Ruida Wang, Jerry Huang, Pengcheng Wang +3
Equipping Large Language Models (LLMs) to execute reliable multi-step workflows has become a central challenge in artificial intelligence. Despite recent advances in LLMs' agentic…
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Xiangyi Li, Yimin Liu, Wenbo Chen +75
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to m…
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
Xuanqing Liu, Luyang Kong, Wei Niu +6
Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. However, analyzing live d…
CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories
Yijia Xiao, Runhui Wang, Luyang Kong +2
The increasing complexity of computer science research projects demands more effective tools for deploying code repositories. Large Language Models (LLMs), such as Anthropic Claude…