4 papers
Wink: Recovering from Misbehaviors in Coding Agents
Rahul Nanda, Chandra Maddila, Smriti Jha +3
Autonomous coding agents, powered by large language models (LLMs), are increasingly being adopted in the software industry to automate complex engineering tasks. However, these age…
RubberDuckBench: A Benchmark for AI Coding Assistants
Ferida Mohammed, Fatma Ayad, Petros Maniatis +2
Programmers are turning to AI coding assistants to answer questions about their code. Benchmarks are needed to soundly evaluate these systems and understand their performance. To e…
ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers
Hannah Lin, Martin Maas, Maximilian Roquemore +17
This paper introduces ECO, a system that automatically modifies source code to improve performance at scale. ECO overcomes the localization problem by combining fleet-wide continuo…
CRQBench: A Benchmark of Code Reasoning Questions
Elizabeth Dinella, Satish Chandra, Petros Maniatis
Large Language Models have demonstrated exceptional proficiency on coding tasks, but it is challenging to precisely evaluate their code reasoning ability. Existing benchmarks are i…