LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates

Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov

Low-rank adaptation (LoRA) is the standard way to fine-tune large models, yet when its two factors are trained independently, the update ignores the geometry of the low-rank weight…