4 citations · 28 across the 86 of their papers we have counts for
99 papers
From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability
Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash +5
We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised met…
AI Consilium: a Benchmark of LLM Clinical Multidisciplinary Second Opinions
Alexey A. Bokov, Ivan I. Panin, Nikolay B. Nechaev +10
Large language models (LLMs) can generate differential diagnoses for complex cases, but a useful second opinion must also be factually grounded, clinically coherent, actionable, an…
What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling
Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4
Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them stro…
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
Arseny Ivanov, Alexander Kolesov, Alexander Korotin +2
Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, li…
Reasoning on the Simplex: Geometric Fixed-Point Models
Talgat Daulbaev, Ilya Glazkov, Maxim Rakhuba +1
Looped reasoners spend test-time compute by iterating a weight-tied map, but a small residual does not mean the state is a fixed point when that map lives in unconstrained latent s…
Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
Ilya Koziev, Ivan Oseledets
Fast matrix multiplication algorithms keep the product fixed and search for a cheaper way to evaluate it. We instead ask whether a Transformer's learned projections can use a diffe…
LANTERN: Illuminating Hidden Mathematical Knowledge in Language Models
Pavel Tikhonov, Elena Tutubalina, Ivan Oseledets +2
Language models can now prove theorems, but people still decide which problems to pursue. We ask whether a model's internal representations can help identify promising mathematical…
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6
While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams…
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference
Ilya Koziev, Leonid Sinev, Ivan Oseledets
Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple tokens and a verification procedure is designed to preserve t…
Structured Transforms for Low-Overhead Quantization of Language Models
Daria Cherniuk, Alexander Rudikov, Boris Kashin +1
We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient…
EvoMem: Memory-Augmented Evolution for Code Optimization
Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8
Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks an…
Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs
Tianchi Yu, Ivan Oseledets
For low-dimensional problems (), spectral methods can achieve exceptionally high accuracy. For middle-dimensional problems (), spectral methods remain…
Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning
Dmitry Beresnev, Vladimir Makharev, Roman Khalikov +2
Many reasoning tasks are not well described by a single left-to-right chain: a solver may need to pursue a plausible branch, observe delayed failure, and return to the latest prefi…
TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
QuocViet Pham, Elvir Karimov, Andrey Galichin +1
LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to c…
LLM-Guided Evolution for Medical Decision Pipelines
Ivan Sviridov, Artem Oskin, Ivan Panin +4
Adapting large language models (LLMs) to clinical workflows often requires costly fine-tuning or manual prompt and pipeline engineering. We study LLM-guided MAP-Elites evolution as…
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze +7
Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practi…
Harnessing non-adversarial robustness in large language models
Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin +6
The work presents an approach for addressing the challenge of robustness in Large Language Models (LLMs) to alterations and potential errors caused by semantically similar but text…
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3
Log-likelihood is a standard metric for evaluating generative models. Unfortunately, in contrast to autoregressive models (ARMs), discrete diffusion models generally do not admit e…
OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models
Ali Aliev, Kamil Garifullin, Nikolay Yudin +5
In a rapidly growing field of model training there is a constant practical interest in parameter-efficient fine-tuning and various techniques that use a small amount of training da…
Marchuk: Efficient Global Weather Forecasting from Mid-Range to Sub-Seasonal Scales via Flow Matching
Arsen Kuzhamuratov, Mikhail Zhirnov, Andrey Kuznetsov +2
Accurate subseasonal weather forecasting remains a major challenge due to the inherently chaotic nature of the atmosphere, which limits the predictive skill of conventional models…
LLM-Guided Evolutionary Search for Algebraic T-Count Optimization
Daniil Fisher, Valentin Khrulkov, Mikhail Saygin +2
T-count minimization is an NP-hard problem that arises in fault-tolerant quantum compilation. In the parity-matrix representation, which captures the non-Clifford part of a quantum…
Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
Anton Korznikov, Andrey Galichin, Alexey Dontsov +3
Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Rece…
IDLM: Inverse-distilled Diffusion Language Models
David Li, Nikita Gushchin, Dmitry Abulkhanov +4
Diffusion Language Models (DLMs) have recently achieved strong results in text generation. However, their multi-step sampling leads to slow inference, limiting practical use. To ad…
ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search
Alexey Kravatskiy, Valentin Khrulkov, Ivan Oseledets
LLM-guided evolutionary computation, most notably AlphaEvolve, has been remarkably successful in discovering novel mathematical constructions by solving challenging optimization pr…