Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning

Mohamed Elsayed, A. Rupam Mahmood

Modern representation learning methods often struggle to adapt quickly under non-stationarity because they suffer from catastrophic forgetting and decaying plasticity. Such problem…