5 papers
PIPer: On-Device Environment Setup via Online Reinforcement Learning
Alexander Kovrigin, Aleksandra Eliseeva, Konstantin Grotov +2
Environment setup-the process of configuring the system to work with a specific software project-represents a persistent challenge in Software Engineering (SE). Automated environme…
EnvBench: A Benchmark for Automated Environment Setup
Aleksandra Eliseeva, Alexander Kovrigin, Ilia Kholkin +2
Recent advances in Large Language Models (LLMs) have enabled researchers to focus on practical repository-level tasks in software engineering domain. In this work, we consider a co…
Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings
Petr Tsvetkov, Aleksandra Eliseeva, Danny Dig +4
When a Commit Message Generation (CMG) system is integrated into the IDEs and other products at JetBrains, we perform online evaluation based on user acceptance of the generated me…
Long Code Arena: a Set of Benchmarks for Long-Context Code Models
Egor Bogomolov, Aleksandra Eliseeva, Timur Galimzyanov +8
Nowadays, the fields of code and natural language processing are evolving rapidly. In particular, models become better at processing long context windows - supported context sizes…
On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing
Alexander Kovrigin, Aleksandra Eliseeva, Yaroslav Zharov +1
Recent advancements in code-fluent Large Language Models (LLMs) enabled the research on repository-level code editing. In such tasks, the model navigates and modifies the entire co…