activity
20202026
most citedOut of the BLEU: how should we assess quality of the Code Generation models?

115 citations · 195 across the 32 of their papers we have counts for

collaborators
Showing 2024 · cs.SEShow all

5 papers · 2 filters

cs.SE2024★ 1 cited

Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code

Timur Galimzyanov, Sergey Titov, Yaroslav Golubev +1

This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses…

cs.SE2024★ 56 cited

EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs

Dorin Pomian, Abhiram Bellur, Malinda Dilhara +5

Excessively long methods, loaded with multiple responsibilities, are challenging to understand, debug, reuse, and maintain. The solution lies in the widely recognized Extract Metho…

cs.SE2024

Kotlin ML Pack: Technical Report

Sergey Titov, Mikhail Evtikhiev, Anton Shapkin +7

In this technical report, we present three novel datasets of Kotlin code: KStack, KStack-clean, and KExercises. We also describe the results of fine-tuning CodeLlama and DeepSeek m…

cs.SE2024

Tool-Augmented LLMs as a Universal Interface for IDEs

Yaroslav Zharov, Yury Khudyakov, Evgeniia Fedotova +2

Modern-day Integrated Development Environments (IDEs) have come a long way from the early text editing utilities to the complex programs encompassing thousands of functions to help…

cs.SE2024★ 2 cited

Together We Go Further: LLMs and IDE Static Analysis for Extract Method Refactoring

Dorin Pomian, Abhiram Bellur, Malinda Dilhara +4

Long methods that encapsulate multiple responsibilities within a single method are challenging to maintain. Choosing which statements to extract into new methods has been the targe…