Decentralized Relaxed Smooth Optimization with Gradient Descent Methods

Zhanhong Jiang, Aditya Balu, Soumik Sarkar

$L_0$-smoothness, which has been pivotal to advancing decentralized optimization theory, is often fairly restrictive for modern tasks like deep learning. The recent advent of relax…