SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

Omer Tariq, Syed Muhammad Raza, Jeongbae Son

Distilling a fine-tuned teacher into a LoRA-adapted student is a standard recipe for parameter-efficient compression, but output-level KD does not explicitly control which rank-$r$…