4 papers
Selective Underfitting in Diffusion Models
Kiwhan Song, Jaeyeon Kim, Sitan Chen +3
Diffusion models have emerged as the principal paradigm for generative modeling across various domains. During training, they learn the score function, which in turn is used to gen…
Any-Order Flexible Length Masked Diffusion
Jaeyeon Kim, Lee Cheuk-Kit, Carles Domingo-Enrich +5
Masked diffusion models (MDMs) have recently emerged as a promising alternative to autoregressive models over discrete domains. MDMs generate sequences in an any-order, parallel fa…
LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)
Junsu Kim, Jaeyeon Kim, Ernest K. Ryu
Low-rank adaptation (LoRA) has become a standard approach for fine-tuning large foundation models. However, our theoretical understanding of LoRA remains limited as prior analyses…
Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
Jaeyeon Kim, Kulin Shah, Vasilis Kontonis +2
In recent years, masked diffusion models (MDMs) have emerged as a promising alternative approach for generative modeling over discrete domains. Compared to autoregressive models (A…