4 papers
ProgramBench: Can Language Models Rebuild Programs From Scratch?
John Yang, Kilian Lieret, Jeffrey Ma +9
Turning ideas into full software projects from scratch has become a popular use case for language models. Agents are being deployed to seed, maintain, and grow codebases over exten…
Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?
Parth Thakkar, Ankush Agarwal, Prasad Kasu +2
While Multi-modal Large Language Models (MLLMs) have shown impressive capabilities in document understanding tasks, their ability to locate and reason about fine-grained details wi…
Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback
Chandra Maddila, Adam Tait, Claire Chang +21
Aim: With the advent of LLMs, sophisticated agentic program repair has become viable at large organizations with large codebases. In this work, we develop an Engineering Agent that…
AI-Assisted Fixes to Code Review Comments at Scale
Chandra Maddila, Negar Ghorbani, James Saindon +7
Aim. There are 10s of thousands of code review comments each week at Meta. We developed Metamate for Code Review (MetaMateCR) that provides AI-assisted fixes for reviewer comments…