115 citations · 386 across the 61 of their papers we have counts for
70 papers
Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required
Egor Shibaev, Vera Kudrevskaia, Timur Galimzyanov +9
Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as evidence of broad coding capability,…
Multi-Agent Coordinated Rename Refactoring
Abhiram Bellur, Mohammed Raihan Ullah, Fraol Batole +9
The primary value of AI agents in software development lies in their ability to extend the developer's capacity for reasoning and action, not to supplant human involvement. To show…
Observing Fine-Grained Changes in Jupyter Notebooks During Development Time
Sergey Titov, Konstantin Grotov, Cristina Sarasua +5
In software engineering research, the analysis of fine-grained logs led to significant innovations in areas such as refactoring, security, and code completion. However, even though…
Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring
Abhiram Bellur, Fraol Batole, Mohammed Raihan Ullah +12
MOVEMETHOD is a hallmark refactoring. Despite a plethora of research tools that recommend which methods to move and where, these recommendations do not align with how expert develo…
In-IDE Programming Courses: Learning Software Development in a Real-World Setting
Anastasiia Birillo, Ilya Vlasov, Katsiaryna Dzialets +2
While learning programming languages is crucial for software engineers, mastering the necessary tools is equally important. To facilitate this, JetBrains recently released the JetB…
Debug Smarter, Not Harder: AI Agents for Error Resolution in Computational Notebooks
Konstantin Grotov, Artem Borzilov, Maksim Krivobok +2
Computational notebooks became indispensable tools for research-related development, offering unprecedented interactivity and flexibility in the development process. However, these…
One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks
Anastasiia Birillo, Elizaveta Artser, Anna Potriasaeva +6
Students often struggle with solving programming problems when learning to code, especially when they have to do it online, with one of the most common disadvantages of working onl…
Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings
Petr Tsvetkov, Aleksandra Eliseeva, Danny Dig +4
When a Commit Message Generation (CMG) system is integrated into the IDEs and other products at JetBrains, we perform online evaluation based on user acceptance of the generated me…
Developer Needs and Feasible Features for AI Assistants in IDEs
Agnia Sergeyuk, Ekaterina Koshchenko, Ilya Zakharov +2
Despite the increasing presence of AI assistants in Integrated Development Environments (IDEs), it remains unclear what different groups of developers actually need from these tool…
Assessing Consensus of Developers' Views on Code Readability
Agnia Sergeyuk, Olga Lvova, Sergey Titov +3
The rapid rise of Large Language Models (LLMs) has changed software development, with tools like Copilot, JetBrains AI Assistant, and others boosting developers' productivity. Howe…
Long Code Arena: a Set of Benchmarks for Long-Context Code Models
Egor Bogomolov, Aleksandra Eliseeva, Timur Galimzyanov +8
Nowadays, the fields of code and natural language processing are evolving rapidly. In particular, models become better at processing long context windows - supported context sizes…
On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing
Alexander Kovrigin, Aleksandra Eliseeva, Yaroslav Zharov +1
Recent advancements in code-fluent Large Language Models (LLMs) enabled the research on repository-level code editing. In such tasks, the model navigates and modifies the entire co…
Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward
Agnia Sergeyuk, Yaroslav Golubev, Timofey Bryksin +1
Context. The last several years saw the emergence of AI assistants for code - multi-purpose AI-based helpers in software engineering. As they become omnipresent in all aspects of s…
EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs
Dorin Pomian, Abhiram Bellur, Malinda Dilhara +5
Excessively long methods, loaded with multiple responsibilities, are challenging to understand, debug, reuse, and maintain. The solution lies in the widely recognized Extract Metho…
Kotlin ML Pack: Technical Report
Sergey Titov, Mikhail Evtikhiev, Anton Shapkin +7
In this technical report, we present three novel datasets of Kotlin code: KStack, KStack-clean, and KExercises. We also describe the results of fine-tuning CodeLlama and DeepSeek m…
Full Line Code Completion: Bringing AI to Desktop
Anton Semenkin, Vitaliy Bibaev, Yaroslav Sokolov +14
In recent years, several industrial solutions for the problem of multi-token code completion appeared, each making a great advance in the area but mostly focusing on cloud-based ru…
Untangling Knots: Leveraging LLM for Error Resolution in Computational Notebooks
Konstantin Grotov, Sergey Titov, Yaroslav Zharov +1
Computational notebooks became indispensable tools for research-related development, offering unprecedented interactivity and flexibility in the development process. However, these…
Clustering MOOC Programming Solutions to Diversify Their Presentation to Students
Elizaveta Artser, Anastasiia Birillo, Yaroslav Golubev +4
In many MOOCs, whenever a student completes a programming task, they can see previous solutions of other students to find potentially different ways of solving the problem and to l…
Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example
Malinda Dilhara, Abhiram Bellur, Timofey Bryksin +1
Software developers often repeat code changes, known as "code change patterns" (CPATs), within and across projects. Automating these CPATs accelerates development, but current Tran…
Reassessing Java Code Readability Models with a Human-Centered Approach
Agnia Sergeyuk, Olga Lvova, Sergey Titov +4
To ensure that Large Language Models (LLMs) effectively support user productivity, they need to be adjusted. Existing Code Readability (CR) models can guide this alignment. However…
Together We Go Further: LLMs and IDE Static Analysis for Extract Method Refactoring
Dorin Pomian, Abhiram Bellur, Malinda Dilhara +4
Long methods that encapsulate multiple responsibilities within a single method are challenging to maintain. Choosing which statements to extract into new methods has been the targe…
Dynamic Retrieval-Augmented Generation
Anton Shapkin, Denis Litvinov, Yaroslav Zharov +3
Current state-of-the-art large language models are effective in generating high-quality text and encapsulating a broad spectrum of world knowledge. These models, however, often hal…
From Commit Message Generation to History-Aware Commit Message Completion
Aleksandra Eliseeva, Yaroslav Sokolov, Egor Bogomolov +3
Commit messages are crucial to software development, allowing developers to track changes and collaborate effectively. Despite their utility, most commit messages lack important in…
Overcoming the Mental Set Effect in Programming Problem Solving
Agnia Sergeyuk, Sergey Titov, Yaroslav Golubev +1
This paper adopts a cognitive psychology perspective to investigate the recurring mistakes in code resulting from the mental set (Einstellung) effect. The Einstellung effect is the…