3 citations · 3 across the 13 of their papers we have counts for
18 papers
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov +6
Prompt-based adaptation offers several ways to specialize a frozen language model, ranging from natural-language prompt optimization to learned continuous prompts. These methods ar…
Multi-Stream Residual Computation in Diffusion Language Models
Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1
Diffusion language models reconstruct masked tokens in parallel, offering an alternative to autoregressive generation. Yet their Transformer backbones typically carry visible evide…
Monarch-Muon: Structure-Aware Muon for Fast and Memory-Efficient Pretraining
Alexander Miasnikov, Ruslan Kabirov, Gleb Molodtsov +7
Training large language models is constrained by both wall-clock time and memory footprint, and recent matrix-aware optimizers sharpen this tradeoff: Muon can improve optimization…
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2
Block diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both memory capacity and generation speed…
Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
Ekaterina Alimaskina, Denis Shveykin, Gleb Molodtsov +3
Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the res…
Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
Ekaterina Alimaskina, Gleb Molodtsov, Aleksandr Beznosikov
Hyper-Connections (HC) replace the single Transformer residual stream with multiple streams, introducing a permutation symmetry over stream indices. We study how this symmetry is r…
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Artur Zagitov, Alexander Miasnikov, Maxim Krutikov +5
Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, off…
Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
Ekaterina Alimaskina, Darya Rudas, Denis Shveykin +3
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit i…
Hierarchical Mixture-of-Experts with Two-Stage Optimization
Gleb Molodtsov, Alexander Miasnikov, Aleksandr Beznosikov
Sparse Mixture-of-Experts (MoE) models scale capacity by routing each token to a small subset of experts. However, their routers exhibit a fundamental trade-off: strong load balanc…
HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov
Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activatio…
Shuffling Heuristic in Variational Inequalities: Establishing New Convergence Guarantees
Daniil Medyakov, Gleb Molodtsov, Grigoriy Evseev +2
Variational inequalities have gained significant attention in machine learning and optimization research. While stochastic methods for solving these problems typically assume indep…
Robust autobidding for noisy conversion prediction models
Andrey Pudovikov, Alexandra Khirianova, Ekaterina Solodneva +4
Managing millions of digital auctions is an essential task for modern advertising auction systems. The main approach to managing digital auctions is an autobidding approach, which…
Communication-Efficient Federated Learning with Adaptive Number of Participants
Sergey Skorik, Vladislav Dorofeev, Gleb Molodtsov +4
Rapid scaling of deep learning models has enabled performance gains across domains, yet it introduced several challenges. Federated Learning (FL) has emerged as a promising framewo…
Sign-SGD via Parameter-Free Optimization
Daniil Medyakov, Sergey Stanko, Gleb Molodtsov +4
Large language models have achieved major advances across domains, yet training them remains extremely resource-intensive. We revisit Sign-SGD, which serves both as a memory-effici…
Bant: Byzantine Antidote via Trial Function and Trust Scores
Gleb Molodtsov, Daniil Medyakov, Sergey Skorik +6
Recent advancements in machine learning have improved performance while also increasing computational demands. While federated and distributed setups address these issues, their st…
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling
Daniil Medyakov, Gleb Molodtsov, Savelii Chezhegov +2
Stochastic optimization algorithms are widely used for machine learning with large-scale data. However, their convergence often suffers from non-vanishing variance. Variance Reduct…
Effective Method with Compression for Distributed and Federated Cocoercive Variational Inequalities
Daniil Medyakov, Gleb Molodtsov, Aleksandr Beznosikov
Variational inequalities as an effective tool for solving applied problems, including machine learning tasks, have been attracting more and more attention from researchers in recen…
Optimal Data Splitting in Distributed Optimization for Machine Learning
Daniil Medyakov, Gleb Molodtsov, Aleksandr Beznosikov +1
The distributed optimization problem has become increasingly relevant recently. It has a lot of advantages such as processing a large amount of data in less time compared to non-di…