9 papers
Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?
Alexander Shabalin, Simon Elistratov, Viacheslav Meshchaninov +2
Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains challenging. We investigate why Gauss…
Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation
Alexander Shabalin, Viacheslav Meshchaninov, Dmitry Vetrov
Diffusion models have achieved state-of-the-art performance in generating images, audio, and video, but their adaptation to text remains challenging due to its discrete nature. Pri…
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4
Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising e…
Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling
Viacheslav Meshchaninov, Egor Chimbulatov, Alexander Shabalin +2
Autoregressive language models dominate modern text generation, yet their sequential nature introduces fundamental limitations: decoding is slow, and maintaining global coherence r…
Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion
Evgeniia Vu, Andrei Boiarov, Dmitry Vetrov
Generating co-speech gestures in real time requires both temporal coherence and efficient sampling. We introduce a novel framework for streaming gesture generation that extends Rol…
Diffusion on language model encodings for protein sequence generation
Viacheslav Meshchaninov, Pavel Strashnov, Andrey Shevtsov +4
Protein sequence design has seen significant advances through discrete diffusion and autoregressive approaches, yet the potential of continuous diffusion remains underexplored. Her…