5 citations · 6 across the 19 of their papers we have counts for
4 papers · 1 filter
Who to Trust? Aggregating Client Predictions in Federated Distillation
Viktor Kovalchuk, Denis Son, Arman Bolatov +6
Under data heterogeneity (e.g., ), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combinatio…
Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
Saleh Vatan Khah, Savelii Chezhegov, Shahrokh Farahmand +2
Gradient clipping is a fundamental tool in Deep Learning, improving the high-probability convergence of stochastic first-order methods like SGD, AdaGrad, and Adam under heavy-taile…
High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and -Smoothness
Savelii Chezhegov, Taha El Bakkali El Kadi, Aleksandr Beznosikov +2
Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees over stochastic gradient descent (SGD) und…
Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
Rustem Islamov, Samuel Horvath, Aurelien Lucchi +2
Strong Differential Privacy (DP) and Optimization guarantees are two desirable properties for a method in Federated Learning (FL). However, existing algorithms do not achieve both…