Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

Dongjun Kim, Adrian de Wynter, Huancheng Chen +2

While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining. Existing forgetting aware…