NOVAK: Unified adaptive optimizer for deep neural networks

Sergii Kavun

This work introduces NOVAK, a modular gradient-based optimization algorithm that integrates adaptive moment estimation, rectified learning-rate scheduling, decoupled weight regular…