6 papers
Multi-Stream Residual Computation in Diffusion Language Models
Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1
Diffusion language models reconstruct masked tokens in parallel, offering an alternative to autoregressive generation. Yet their Transformer backbones typically carry visible evide…
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2
Block diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both memory capacity and generation speed…
Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
Ekaterina Alimaskina, Denis Shveykin, Gleb Molodtsov +3
Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the res…
Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
Ekaterina Alimaskina, Gleb Molodtsov, Aleksandr Beznosikov
Hyper-Connections (HC) replace the single Transformer residual stream with multiple streams, introducing a permutation symmetry over stream indices. We study how this symmetry is r…
Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
Ekaterina Alimaskina, Darya Rudas, Denis Shveykin +3
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit i…
Metropolis-Scale Road Network Datasets for Fine-Grained Urban Traffic Modeling
Fedor Velikonivtsev, Oleg Platonov, Ekaterina Alimaskina +2
Modeling traffic dynamics is a critical challenge for urban computing, with applications from real-time traffic management to infrastructure planning. However, progress in this are…