31 citations · 80 across the 14 of their papers we have counts for
14 papers
AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders
Georgii Aparin, Tasnima Sadekova, Alexey Rukhovich +5
Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whis…
Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story
Vladislav Pedashenko, Laida Kushnareva, Yana Khassan Nibal +5
Intrinsic dimension (ID) is an important tool in modern LLM analysis, informing studies of training dynamics, scaling behavior, and dataset structure, yet its textual determinants…
Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders
Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina +5
Artificial Text Detection (ATD) is becoming increasingly important with the rise of advanced Large Language Models (LLMs). Despite numerous efforts, no single algorithm performs co…
Quantifying Logical Consistency in Transformers via Query-Key Alignment
Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva +4
Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks, yet their ability to perform multi-step logical reasoning remain…
Robust AI-Generated Text Detection by Restricted Embeddings
Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva +4
Growing amount and quality of AI-generated texts makes detecting such content more difficult. In most real-world scenarios, the domain (style and topic) of generated data and the g…
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
Eduard Tulchinskii, Laida Kushnareva, Kristian Kuznetsov +5
Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer internally. We expose this latent kn…
Improving Interpretability and Robustness for the Detection of AI-Generated Images
Tatiana Gaintseva, Laida Kushnareva, German Magai +5
With growing abilities of generative models, artificial content detection becomes an increasingly important and difficult task. However, all popular approaches to this problem suff…
AI-generated text boundary detection with RoFT
Laida Kushnareva, Tatiana Gaintseva, German Magai +6
Due to the rapid development of large language models, people increasingly often encounter texts that may start as written by a human but continue as machine-generated. Detecting t…
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5
Rapidly increasing quality of AI-generated content makes it difficult to distinguish between human and AI-generated texts, which may lead to undesirable consequences for society. T…
Topological Data Analysis for Speech Processing
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5
We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT. To this end, we introduce a number of topo…
Betti numbers of attention graphs is all you really need
Laida Kushnareva, Dmitri Piontkovski, Irina Piontkovskaya
We apply methods of topological analysis to the attention graphs, calculated on the attention heads of the BERT model ( arXiv:1810.04805v2 ). Our research shows that the classifier…
Acceptability Judgements via Examining the Topology of Attention Maps
Daniil Cherniavskii, Eduard Tulchinskii, Vladislav Mikhailov +7
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP. However, the ability of the attention heads to judge the grammatical acce…
Artificial Text Detection via Examining the Topology of Attention Maps
Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov +6
The impressive capabilities of recent generative models to create texts that are challenging to distinguish from the human-written ones can be misused for generating fake news, pro…
Category-Learning with Context-Augmented Autoencoder
Denis Kuzminykh, Laida Kushnareva, Timofey Grigoryev +1
Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. Biological neural networks are known to solve this problem…