3 papers
cs.LG2025
Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares
Lachlan Ewen MacDonald, Hancheng Min, Leandro Palma +3
Classical optimisation theory guarantees monotonic objective decrease for gradient descent (GD) when employed in a small step size, or ``stable", regime. In contrast, gradient desc…
math.PR2025
Disintegration theorem for multifunctions, with applications to empirical Wasserstein distances and average-case statistical bounds
James Allen Fill, Lachlan Ewen MacDonald
We prove a generalisation of the disintegration theorem to the setting of multifunctions between Polish probability spaces. Whereas the classical disintegration theorem guarantees…
cs.LG2025
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization
Ziqing Xu, Hancheng Min, Lachlan Ewen MacDonald +4
Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pre-trained models, there is little theoretical understanding of how first-order methods with carefully c…