When Benchmarks Talk: Re-Evaluating Code LLMs with Interactive Feedback

Jane Pan, Ryan Shar, Jacob Pfau +3

Programming is a fundamentally interactive process, yet coding assistants are often evaluated using static benchmarks that fail to measure how well models collaborate with users. W…