Bilinear Optimization Divergence: Diagnosing Factor-Constrained LoRA Continual Learning

YongShun Wang, JianLin Su, Yong Ma

Orthogonality in a LoRA factor does not by itself specify what the composed update protects: the answer depends on the task-start state, the parameterization, and the realized opti…