EditFlow: Benchmarking and Optimizing Code Edit Recommendation Systems via Reconstruction of Developer Flows

Abstract

Large language models (LLMs) for code editing have achieved remarkable progress, yet recent empirical studies reveal a fundamental disconnect between technical accuracy and developer productivity. Despite their strong benchmark performance, developers complete tasks 19% slower when using AI assistance, with over 68.81% of recommendations disrupting their mental flow. This misalignment stems from the use of static commit snapshots that lack temporal information, causing models to optimize for end results rather than the incremental, context-sensitive steps that align with developers' natural reasoning process. To bridge this gap, we present EditFlow, which benchmarks and optimizes subsequent code edit recommendation systems through the reconstruction of developer editing flows. EditFlow addresses three key challenges. First, collecting edit-order data that reflects developers' flow is inherently difficult: manual annotation introduces prohibitive overhead, while development logs capture only single trajectories instead of all plausible editing flows. Second, benchmarking recommendation performance against developers' ongoing editing flow requires a digital-twin-like simulation that can faithfully simulate the editing process. Third, existing heterogeneous systems vary drastically in scale and architecture, posing challenges for developing a unified optimization strategy that endows all models with mental-flow awareness regardless of design or capability. ......

This paper has been accepted to the Proceedings of the ACM on Programming Languages (OOPSLA 2026). (Proc. ACM Program. Lang., Vol. 10, OOPSLA1)

References in corpus (15)

• "It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers
• CODIT: Code Editing with Tree-Based Neural Models
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• CodeEditor: Learning to Edit Source Code with Pre-trained Models
• Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
• OlympicArena Medal Ranks: Who Is the Most Intelligent AI So Far?
• CoEdPilot: Recommending Code Edits with Learned Prior Edit Relevance, Project-wise Awareness, and Interactive Nature
• The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study
• Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
• Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding
• Generating Project-Specific Test Cases with Requirement Validation Intention
• Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
• Learning Project-wise Subsequent Code Edits via Interleaving Neural-based Induction and Tool-based Deduction