3 citations · 3 across the 13 of their papers we have counts for
8 papers · 2 filters
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov +6
Prompt-based adaptation offers several ways to specialize a frozen language model, ranging from natural-language prompt optimization to learned continuous prompts. These methods ar…
Multi-Stream Residual Computation in Diffusion Language Models
Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1
Diffusion language models reconstruct masked tokens in parallel, offering an alternative to autoregressive generation. Yet their Transformer backbones typically carry visible evide…
Monarch-Muon: Structure-Aware Muon for Fast and Memory-Efficient Pretraining
Alexander Miasnikov, Ruslan Kabirov, Gleb Molodtsov +7
Training large language models is constrained by both wall-clock time and memory footprint, and recent matrix-aware optimizers sharpen this tradeoff: Muon can improve optimization…
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2
Block diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both memory capacity and generation speed…
Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
Ekaterina Alimaskina, Gleb Molodtsov, Aleksandr Beznosikov
Hyper-Connections (HC) replace the single Transformer residual stream with multiple streams, introducing a permutation symmetry over stream indices. We study how this symmetry is r…
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Artur Zagitov, Alexander Miasnikov, Maxim Krutikov +5
Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, off…
Hierarchical Mixture-of-Experts with Two-Stage Optimization
Gleb Molodtsov, Alexander Miasnikov, Aleksandr Beznosikov
Sparse Mixture-of-Experts (MoE) models scale capacity by routing each token to a small subset of experts. However, their routers exhibit a fundamental trade-off: strong load balanc…
HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov
Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activatio…