Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Gradient Regularized Natural Gradients
Satya Prakash Dash, Hossein Abdi, Wei Pan +2
Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initi…
cs.LG2025
Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
Hossein Abdi, Mingfei Sun, Wei Pan
Vision-language pre-trained models, such as CLIP, have established new benchmarks in multimodal data mining. In such models, few-shot fine-tuning is a major challenge to achieve op…
cs.LG2024
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
Hossein Abdi, Mingfei Sun, Andi Zhang +2
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…