2 citations · 2 across the 4 of their papers we have counts for
5 papers
Rotated Manifold Optimization for Low-Rank Adaptation
Yuhui Ding, Javier Zazo, James Hensman
We propose a novel optimizer for low-rank adaptation (LoRA) that explicitly incorporates the gauge symmetry of low-rank factorization. Our optimizer extends recent matrix optimizer…
Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
Grigory Bartosh, Teodora Pandeva, Sushrut Karmalkar +1
Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes…
ARO: A New Lens On Matrix Optimization For Large Models
Wenbo Gong, Javier Zazo, Qijun Luo +3
Matrix-based optimizers have attracted growing interest for improving LLM training efficiency, with significant progress centered on orthogonalization/whitening based methods. Whil…
Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
Iskander Azangulov, Teodora Pandeva, Niranjani Prasad +2
Masked diffusion models (MDMs) offer a compelling alternative to autoregressive models (ARMs) for discrete text generation because they enable parallel token sampling, rather than…
A Fourier Space Perspective on Diffusion Models
Fabian Falck, Teodora Pandeva, Kiarash Zahirnia +5
Diffusion models are state-of-the-art generative models on data modalities such as images, audio, proteins and materials. These modalities share the property of exponentially decay…