3 citations · 8 across the 11 of their papers we have counts for
11 papers
CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure
Eric Ding, Byungsoo Oh, Bhaskar Kataria +8
Evaluative claims about LLM infrastructure -- ``workload X is fastest on hardware Y with software Z'' -- depend on a complex configuration space spanning hardware accelerators, int…
OmniCode: A Benchmark for Evaluating Software Engineering Agents
Atharv Sonwane, Eng-Shen Tu, Wei-Chung Lu +11
LLM-powered coding agents are redefining how real-world software is developed. To drive the research towards better coding agents, we require challenging benchmarks that can rigoro…
Gistify! Codebase-Level Understanding via Runtime Execution
Hyunji Lee, Minseon Kim, Chinmay Singh +10
As coding agents are increasingly deployed in large codebases, the need to automatically design challenging, codebase-level evaluation is central. We propose Gistify, a task where…
BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
Atharv Sonwane, Isadora White, Hyunji Lee +8
High quality bugs are key to training the next generation of language model based software engineering (SWE) agents. We introduce a novel method for synthetic generation of difficu…
MASAI: Modular Architecture for Software-engineering AI Agents
Daman Arora, Atharv Sonwane, Nalin Wadhwa +5
A common method to solve complex problems in software engineering, is to divide the problem into multiple sub-problems. Inspired by this, we propose a Modular Architecture for Soft…
Frustrated with Code Quality Issues? LLMs can Help!
Nalin Wadhwa, Jui Pradhan, Atharv Sonwane +5
As software projects progress, quality of code assumes paramount importance as it affects reliability, maintainability and security of software. For this reason, static analysis to…
CodePlan: Repository-level Coding using LLMs and Planning
Ramakrishna Bairi, Atharv Sonwane, Aditya Kanade +6
Software engineering activities such as package migration, fixing errors reports from static analysis or testing, and adding type annotations or other specifications to a codebase,…
StaticFixer: From Static Analysis to Static Repair
Naman Jain, Shubham Gandhi, Atharv Sonwane +5
Static analysis tools are traditionally used to detect and flag programs that violate properties. We show that static analysis tools can also be used to perturb programs that satis…
Policy-Guided Lazy Search with Feedback for Task and Motion Planning
Mohamed Khodeir, Atharv Sonwane, Ruthrash Hari +1
PDDLStream solvers have recently emerged as viable solutions for Task and Motion Planning (TAMP) problems, extending PDDL to problems with continuous action spaces. Prior work has…
Solving Visual Analogies Using Neural Algorithmic Reasoning
Atharv Sonwane, Gautam Shroff, Lovekesh Vig +2
We consider a class of visual analogical reasoning problems that involve discovering the sequence of transformations by which pairs of input/output images are related, so as to ana…
Using Program Synthesis and Inductive Logic Programming to solve Bongard Problems
Atharv Sonwane, Sharad Chitlangia, Tirtharaj Dash +3
The ability to recognise and make analogies is often used as a measure or test of human intelligence. The ability to solve Bongard problems is an example of such a test. It has als…