2 papers
cs.LG2026
Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning
Nan Chen, Soledad Villar, Soufiane Hayou
Low-Rank Adaptation (LoRA) is a standard tool for parameter-efficient finetuning of large models. While it induces a small memory footprint, its training dynamics can be surprising…
cs.LG2025
On Transferring Transferability: Towards a Theory for Size Generalization
Eitan Levin, Yuxin Ma, Mateo Díaz +1
Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs a…