31 citations · 32 across the 16 of their papers we have counts for
16 papers
AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders
Georgii Aparin, Tasnima Sadekova, Alexey Rukhovich +5
Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whis…
Gamayun's Path to Multilingual Mastery: Cost-Efficient Training of a 1.5B-Parameter LLM
Alexander Podolskiy, Semen Molokov, Timofey Gerasin +14
We present Gamayun, a 1.5B-parameter multilingual language model trained entirely from scratch on 2.5T tokens. Designed for efficiency and deployment in resource-constrained enviro…
Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story
Vladislav Pedashenko, Laida Kushnareva, Yana Khassan Nibal +5
Intrinsic dimension (ID) is an important tool in modern LLM analysis, informing studies of training dynamics, scaling behavior, and dataset structure, yet its textual determinants…
YABLoCo: Yet Another Benchmark for Long Context Code Generation
Aidar Valeev, Roman Garaev, Vadim Lomshakov +3
Large Language Models demonstrate the ability to solve various programming tasks, including code generation. Typically, the performance of LLMs is measured on benchmarks with small…
LAMeD: LLM-generated Annotations for Memory Leak Detection
Ekaterina Shemetova, Ilya Shenbin, Ivan Smirnov +5
Static analysis tools are widely used to detect software bugs and vulnerabilities but often struggle with scalability and efficiency in complex codebases. Traditional approaches re…
Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders
Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina +5
Artificial Text Detection (ATD) is becoming increasingly important with the rise of advanced Large Language Models (LLMs). Despite numerous efforts, no single algorithm performs co…
Quantifying Logical Consistency in Transformers via Query-Key Alignment
Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva +4
Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks, yet their ability to perform multi-step logical reasoning remain…
Commute Your Domains: Trajectory Optimality Criterion for Multi-Domain Learning
Alexey Rukhovich, Alexander Podolskiy, Irina Piontkovskaya
In multi-domain learning, a single model is trained on diverse data domains to leverage shared knowledge and improve generalization. The order in which the data from these domains…
Robust AI-Generated Text Detection by Restricted Embeddings
Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva +4
Growing amount and quality of AI-generated texts makes detecting such content more difficult. In most real-world scenarios, the domain (style and topic) of generated data and the g…
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
Eduard Tulchinskii, Laida Kushnareva, Kristian Kuznetsov +5
Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer internally. We expose this latent kn…
Improving Interpretability and Robustness for the Detection of AI-Generated Images
Tatiana Gaintseva, Laida Kushnareva, German Magai +5
With growing abilities of generative models, artificial content detection becomes an increasingly important and difficult task. However, all popular approaches to this problem suff…
Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule
Andrey Bout, Alexander Podolskiy, Sergey Nikolenko +1
Progress in neural grammatical error correction (GEC) is hindered by the lack of annotated training data. Sufficient amounts of high-quality manually annotated data are not availab…
GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding
Konstantin Yakovlev, Alexander Podolskiy, Andrey Bout +2
Grammatical error correction (GEC) is an important NLP task that is currently usually solved with autoregressive sequence-to-sequence models. However, approaches of this class are…
Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval
Konstantin Yakovlev, Gregory Polyakov, Ilseyar Alimova +4
A recent trend in multimodal retrieval is related to postprocessing test set results via the dual-softmax loss (DSL). While this approach can bring significant improvements, it usu…
AI-generated text boundary detection with RoFT
Laida Kushnareva, Tatiana Gaintseva, German Magai +6
Due to the rapid development of large language models, people increasingly often encounter texts that may start as written by a human but continue as machine-generated. Detecting t…
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5
Rapidly increasing quality of AI-generated content makes it difficult to distinguish between human and AI-generated texts, which may lead to undesirable consequences for society. T…