Adaptive Friction in Deep Learning: Enhancing Optimizers with Sigmoid and Tanh Function

Hongye Zheng, Bingxing Wang, Minheng Xiao +3

Adaptive optimizers are pivotal in guiding the weight updates of deep neural networks, yet they often face challenges such as poor generalization and oscillation issues. To counter…