4 papers
asr_eval: Algorithms and tools for multi-reference and streaming speech recognition evaluation
Oleg Sedukhin, Andrey Kostin
We propose several improvements to the speech recognition evaluation. First, we propose a string alignment algorithm that supports both multi-reference labeling, arbitrary-length i…
Pisets: A Robust Speech Recognition System for Lectures and Interviews
Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin +3
This work presents a speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving speech recognition accuracy w…
TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
Elena Bruches, Vadim Alperovich, Dari Baturova +8
While Large Language Models (LLMs) have shown promise in software engineering, their application to unit testing remains largely confined to isolated test generation or oracle pred…
RM -RF: Reward Model for Run-Free Unit Test Evaluation
Elena Bruches, Daniil Grebenkin, Mikhail Klementev +8
We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF pre…