5 papers
Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?
Alexander Shabalin, Simon Elistratov, Viacheslav Meshchaninov +2
Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains challenging. We investigate why Gauss…
Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation
Alexander Shabalin, Viacheslav Meshchaninov, Dmitry Vetrov
Diffusion models have achieved state-of-the-art performance in generating images, audio, and video, but their adaptation to text remains challenging due to its discrete nature. Pri…
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4
Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising e…
Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling
Viacheslav Meshchaninov, Egor Chimbulatov, Alexander Shabalin +2
Autoregressive language models dominate modern text generation, yet their sequential nature introduces fundamental limitations: decoding is slow, and maintaining global coherence r…
TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings
Alexander Shabalin, Viacheslav Meshchaninov, Egor Chimbulatov +6
This paper presents the Text Encoding Diffusion Model (TEncDM), a novel approach to diffusion modeling that operates in the space of pre-trained language model encodings. In contra…