6 papers
Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
Ilya Koziev, Ivan Oseledets
Fast matrix multiplication algorithms keep the product fixed and search for a cheaper way to evaluate it. We instead ask whether a Transformer's learned projections can use a diffe…
Zarya: A Hybrid Autoregressive--Masked Diffusion Language Model with Flexible Training and Dual-Mode Inference
Leonid Sinev, Ilya Koziev, Vladislav Leshchuk
Autoregressive language models (ARMs) are constrained by sequential, left-to-right generation, while masked diffusion models (MDMs) enable parallel decoding but suffer from high co…
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference
Ilya Koziev, Leonid Sinev, Ivan Oseledets
Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple tokens and a verification procedure is designed to preserve t…
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4
Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising e…
Detecting Spelling and Grammatical Anomalies in Russian Poetry Texts
Ilya Koziev
The quality of natural language texts in fine-tuning datasets plays a critical role in the performance of generative models, particularly in computational creativity tasks such as…
Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry
Ilya Koziev
Generative poetry systems require effective tools for data engineering and automatic evaluation, particularly to assess how well a poem adheres to versification rules, such as the…