5 citations · 6 across the 19 of their papers we have counts for
4 papers · 1 filter
High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and -Smoothness
Savelii Chezhegov, Taha El Bakkali El Kadi, Aleksandr Beznosikov +2
Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees over stochastic gradient descent (SGD) und…
Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
Yury Demidovich, Petr Ostroukhov, Grigory Malinovsky +4
Non-convex Machine Learning problems typically do not adhere to the standard smoothness assumption. Based on empirical findings, Zhang et al. (2020b) proposed a more realistic gene…
Methods for Convex -Smooth Optimization: Clipping, Acceleration, and Adaptivity
Eduard Gorbunov, Nazarii Tupitsa, Sayantan Choudhury +4
Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most pop…
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
Ahmad Rammal, Kaja Gruntkowska, Nikita Fedin +2
Byzantine robustness is an essential feature of algorithms for certain distributed optimization problems, typically encountered in collaborative/federated learning. These problems…