1 citations · 3 across the 9 of their papers we have counts for
4 papers · 1 filter
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani +2
AI coding agents have shown great progress on Python software engineering benchmarks like SWE-Bench, and for other languages like Java and C in benchmarks like Multi-SWE-Bench. How…
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
Shashank Kirtania, Param Biyani, Priyanshu Gupta +4
Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from ex…
ConDABench: Interactive Evaluation of Language Models for Data Analysis
Avik Dutta, Priyanshu Gupta, Hosein Hasanbeig +6
Real-world data analysis tasks often come with under-specified goals and unclean data. User interaction is necessary to understand and disambiguate a user's intent, and hence, esse…
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani +2
Software Engineering Agents (SWE agents) can autonomously perform development tasks on benchmarks like SWE Bench, but still face challenges when tackling complex and ambiguous real…