AdaCubic: An Adaptive Cubic Regularization Optimizer for Deep Learning

Ioannis Tsingalis, Constantine Kotropoulos, Corentin Briat

A novel regularization technique, AdaCubic, is proposed that adapts the weight of the cubic term. The heart of AdaCubic is an auxiliary optimization problem with cubic constraints…