6 papers
Little Brains, Big Feats: Exploring Compact Language Models
Dari Baturova, Elena Bruches, Ivan Chernov +3
While large language models have been dominating the research landscape recently, small language models remain highly relevant across various domains; yet, they receive far less at…
RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generation
Ivan Bondarenko, Roman Derunets, Oleg Sedukhin +3
We present our winning system for Task~B (generation with reference passages) in SemEval-2026 Task~8: MTRAGEval. Our method is a heterogeneous ensemble of seven LLMs with two promp…
Pisets: A Robust Speech Recognition System for Lectures and Interviews
Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin +3
This work presents a speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving speech recognition accuracy w…
TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
Elena Bruches, Vadim Alperovich, Dari Baturova +8
While Large Language Models (LLMs) have shown promise in software engineering, their application to unit testing remains largely confined to isolated test generation or oracle pred…
RM -RF: Reward Model for Run-Free Unit Test Evaluation
Elena Bruches, Daniil Grebenkin, Mikhail Klementev +8
We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF pre…
MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks
Artem Chervyakov, Alexander Kharitonov, Pavel Zadorozhny +20
Advancements in LLMs have enhanced task automation in software engineering; however, current evaluations primarily focus on natural language tasks, overlooking code quality. Most b…