4 papers
SpecBench: Evaluating Specification-Level Reasoning for Software Engineering LLM Agents
Grant Hamblin, Kevin Song, Zhanda Zhu +4
Software engineering (SWE) agents are transitioning from code generation to full software development lifecycle automation. A critical phase in this lifecycle is specification desi…
What Limits Agentic Systems Efficiency?
Song Bian, Minghao Yan, Anand Jayarajan +2
Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated strong reasoning capabilities. To further enhance LLM capabilities, recent agentic systems, such…
Aegis: Taxonomy and Optimizations for Overcoming Agent-Environment Failures in LLM Agents
Kevin Song, Anand Jayarajan, Yaoyao Ding +4
Large Language Models (LLMs) agents augmented with domain tools promise to autonomously execute complex tasks requiring human-level intelligence, such as customer service and digit…
Tally: Non-Intrusive Performance Isolation for Concurrent Deep Learning Workloads
Wei Zhao, Anand Jayarajan, Gennady Pekhimenko
GPU underutilization is a significant concern in many production deep learning clusters, leading to prolonged job queues and increased operational expenses. A promising solution to…