3 papers
cs.SE2025
MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks
Artem Chervyakov, Alexander Kharitonov, Pavel Zadorozhny +20
Advancements in LLMs have enhanced task automation in software engineering; however, current evaluations primarily focus on natural language tasks, overlooking code quality. Most b…
cs.CL2025
YABLoCo: Yet Another Benchmark for Long Context Code Generation
Aidar Valeev, Roman Garaev, Vadim Lomshakov +3
Large Language Models demonstrate the ability to solve various programming tasks, including code generation. Typically, the performance of LLMs is measured on benchmarks with small…
cs.CL2025
Code Summarization Beyond Function Level
Vladimir Makharev, Vladimir Ivanov
Code summarization is a critical task in natural language processing and software engineering, which aims to generate concise descriptions of source code. Recent advancements have…