4 papers
Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
Grigory Bartosh, Teodora Pandeva, Sushrut Karmalkar +1
Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes…
ARO: A New Lens On Matrix Optimization For Large Models
Wenbo Gong, Javier Zazo, Qijun Luo +3
Matrix-based optimizers have attracted growing interest for improving LLM training efficiency, with significant progress centered on orthogonalization/whitening based methods. Whil…
Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
Iskander Azangulov, Teodora Pandeva, Niranjani Prasad +2
Masked diffusion models (MDMs) offer a compelling alternative to autoregressive models (ARMs) for discrete text generation because they enable parallel token sampling, rather than…
A Fourier Space Perspective on Diffusion Models
Fabian Falck, Teodora Pandeva, Kiarash Zahirnia +5
Diffusion models are state-of-the-art generative models on data modalities such as images, audio, proteins and materials. These modalities share the property of exponentially decay…