LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models

Hossein Abdi, Mingfei Sun, Andi Zhang +2

Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…