4 citations · 4 across the 7 of their papers we have counts for
10 papers
Context or Knowledge: How LLMs Attribute Their Own Answers
Kirill Afendulev, Alexey Dontsov, Alla Chepurova +2
Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what determines their source report?…
From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability
Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash +5
We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised met…
On-sky performance and prospects of near-infrared CMOS detectors in exoplanet detection: first year of SPIRIT
Clàudia Janó-Muñoz, Matthew J. Hooton, Peter P. Pedersen +8
Ultracool dwarfs are promising targets for detecting and characterising temperate terrestrial exoplanets. Their flux peaks in the near-infrared, beyond the wavelength cut-off of th…
What Attention Recalls and Recurrence Controls in Hybrid Language Models
Kirill Afendulev, Alexey Dontsov, Elena Tutubalina +1
Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill…
Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
Anton Korznikov, Andrey Galichin, Alexey Dontsov +3
Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Rece…
OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features
Anton Korznikov, Andrey Galichin, Alexey Dontsov +3
Sparse autoencoders (SAEs) are a technique for sparse decomposition of neural network activations into human-interpretable features. However, current SAEs suffer from feature absor…
Motivating Next-Gen Accelerators with Flexible (N:M) Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches
Shirin Alanova, Kristina Kazistova, Ekaterina Galaeva +7
The demand for efficient large language model (LLM) inference has intensified the focus on sparsification techniques. While semi-structured (N:M) pruning is well-established for we…
The Rogue Scalpel: Activation Steering Compromises LLM Safety
Anton Korznikov, Andrey Galichin, Alexey Dontsov +3
Activation steering is a promising technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference. It is of…
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
Andrey Galichin, Alexey Dontsov, Polina Druzhinina +4
Recent LLMs like DeepSeek-R1 have demonstrated state-of-the-art performance by integrating deep thinking and complex reasoning during generation. However, the internal mechanisms b…
CLEAR: Character Unlearning in Textual and Visual Modalities
Alexey Dontsov, Dmitrii Korzh, Alexey Zhavoronkin +6
Machine Unlearning (MU) is critical for removing private or hazardous information from deep learning models. While MU has advanced significantly in unimodal (text or vision) settin…