2 citations · 2 across the 4 of their papers we have counts for
3 papers · 1 filter
Rotated Manifold Optimization for Low-Rank Adaptation
Yuhui Ding, Javier Zazo, James Hensman
We propose a novel optimizer for low-rank adaptation (LoRA) that explicitly incorporates the gauge symmetry of low-rank factorization. Our optimizer extends recent matrix optimizer…
ARO: A New Lens On Matrix Optimization For Large Models
Wenbo Gong, Javier Zazo, Qijun Luo +3
Matrix-based optimizers have attracted growing interest for improving LLM training efficiency, with significant progress centered on orthogonalization/whitening based methods. Whil…
Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
Iskander Azangulov, Teodora Pandeva, Niranjani Prasad +2
Masked diffusion models (MDMs) offer a compelling alternative to autoregressive models (ARMs) for discrete text generation because they enable parallel token sampling, rather than…