11 papers
Towards Evaluation of Implicit Software World Models in Coding LLMs
Egor Bogomolov, Yaroslav Zharov
Software engineering, whether performed by humans or by AI agents, requires reasoning about how software behaves. We call the internal model that supports such reasoning the softwa…
Multi-Agent Coordinated Rename Refactoring
Abhiram Bellur, Mohammed Raihan Ullah, Fraol Batole +9
The primary value of AI agents in software development lies in their ability to extend the developer's capacity for reasoning and action, not to supplant human involvement. To show…
PIPer: On-Device Environment Setup via Online Reinforcement Learning
Alexander Kovrigin, Aleksandra Eliseeva, Konstantin Grotov +2
Environment setup-the process of configuring the system to work with a specific software project-represents a persistent challenge in Software Engineering (SE). Automated environme…
The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
Tobias Lindenbauer, Igor Slinko, Ludwig Felder +2
Large Language Model (LLM)-based agents solve complex tasks through iterative reasoning, exploration, and tool-use, a process that can result in long, expensive context histories.…
GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git
Tobias Lindenbauer, Egor Bogomolov, Yaroslav Zharov
Benchmarks for Software Engineering (SE) AI agents, most notably SWE-bench, have catalyzed progress in programming capabilities of AI agents. However, they overlook critical develo…
EnvBench: A Benchmark for Automated Environment Setup
Aleksandra Eliseeva, Alexander Kovrigin, Ilia Kholkin +2
Recent advances in Large Language Models (LLMs) have enabled researchers to focus on practical repository-level tasks in software engineering domain. In this work, we consider a co…