3 citations · 3 across the 13 of their papers we have counts for
10 papers · 1 filter
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov +6
Prompt-based adaptation offers several ways to specialize a frozen language model, ranging from natural-language prompt optimization to learned continuous prompts. These methods ar…
Multi-Stream Residual Computation in Diffusion Language Models
Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1
Diffusion language models reconstruct masked tokens in parallel, offering an alternative to autoregressive generation. Yet their Transformer backbones typically carry visible evide…
Monarch-Muon: Structure-Aware Muon for Fast and Memory-Efficient Pretraining
Alexander Miasnikov, Ruslan Kabirov, Gleb Molodtsov +7
Training large language models is constrained by both wall-clock time and memory footprint, and recent matrix-aware optimizers sharpen this tradeoff: Muon can improve optimization…
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2
Block diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both memory capacity and generation speed…
Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
Ekaterina Alimaskina, Denis Shveykin, Gleb Molodtsov +3
Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the res…
Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
Ekaterina Alimaskina, Gleb Molodtsov, Aleksandr Beznosikov
Hyper-Connections (HC) replace the single Transformer residual stream with multiple streams, introducing a permutation symmetry over stream indices. We study how this symmetry is r…
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Artur Zagitov, Alexander Miasnikov, Maxim Krutikov +5
Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, off…
Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
Ekaterina Alimaskina, Darya Rudas, Denis Shveykin +3
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit i…
Hierarchical Mixture-of-Experts with Two-Stage Optimization
Gleb Molodtsov, Alexander Miasnikov, Aleksandr Beznosikov
Sparse Mixture-of-Experts (MoE) models scale capacity by routing each token to a small subset of experts. However, their routers exhibit a fundamental trade-off: strong load balanc…
HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov
Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activatio…