3 papers
cs.LG2026
Gradient Regularized Natural Gradients
Satya Prakash Dash, Hossein Abdi, Wei Pan +2
Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initi…
cs.LG2025
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
Hossein Abdi, Mingfei Sun, Andi Zhang +2
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…
cs.LG2025
Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
Hossein Abdi, Mingfei Sun, Wei Pan
Vision-language pre-trained models, such as CLIP, have established new benchmarks in multimodal data mining. In such models, few-shot fine-tuning is a major challenge to achieve op…