4 papers
Pisets: A Robust Speech Recognition System for Lectures and Interviews
Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin +3
This work presents a speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving speech recognition accuracy w…
TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
Elena Bruches, Vadim Alperovich, Dari Baturova +8
While Large Language Models (LLMs) have shown promise in software engineering, their application to unit testing remains largely confined to isolated test generation or oracle pred…
RM -RF: Reward Model for Run-Free Unit Test Evaluation
Elena Bruches, Daniil Grebenkin, Mikhail Klementev +8
We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF pre…
MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks
Artem Chervyakov, Alexander Kharitonov, Pavel Zadorozhny +20
Advancements in LLMs have enhanced task automation in software engineering; however, current evaluations primarily focus on natural language tasks, overlooking code quality. Most b…