9 citations · 15 across the 31 of their papers we have counts for
4 papers · 2 filters
Continuous Benchmark Generation for Evaluating Enterprise-scale LLM Agents
Divyanshu Saxena, Rishikesh Maurya, Xiaoxuan Ou +7
The rapid adoption of AI agents across domains has made systematic evaluation crucial for ensuring their usefulness and successful production deployment. Evaluation of AI agents ty…
Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems
Chenyu Zhao, Shenglin Zhang, Zeshun Huang +8
Large language models (LLMs) have shown growing potential in software engineering, yet few benchmarks evaluate their ability to repair software during migration across instruction…
SWE-bench Goes Live!
Linghao Zhang, Shilin He, Chaoyun Zhang +12
The issue-resolving task, where a model generates patches to fix real-world bugs, has emerged as a critical benchmark for evaluating the capabilities of large language models (LLMs…
eARCO: Efficient Automated Root Cause Analysis with Prompt Optimization
Drishti Goel, Raghav Magazine, Supriyo Ghosh +5
Root cause analysis (RCA) for incidents in large-scale cloud systems is a complex, knowledge-intensive task that often requires significant manual effort from on-call engineers (OC…