81 citations · 181 across the 12 of their papers we have counts for
12 papers
Modulated learning for private and distributed regression with just a single sample per client device
Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth +1
This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this on…
Flashback: Understanding and Mitigating Forgetting in Federated Learning
Mohammed Aljahdali, Ahmed M. Abdelmoniem, Marco Canini +1
In Federated Learning (FL), forgetting, or the loss of knowledge across rounds, hampers algorithm convergence, particularly in the presence of severe data heterogeneity among clien…
Convergence of Proximal Point and Extragradient-Based Methods Beyond Monotonicity: the Case of Negative Comonotonicity
Eduard Gorbunov, Adrien Taylor, Samuel Horváth +1
Algorithms for min-max optimization and variational inequalities are often studied under monotonicity assumptions. Motivated by non-monotone machine learning applications, we follo…
Adaptive Learning Rates for Faster Stochastic Gradient Methods
Samuel Horváth, Konstantin Mishchenko, Peter Richtárik
In this work, we propose new adaptive step size strategies that improve several stochastic gradient methods. Our first method (StoPS) is based on the classical Polyak step size (Po…
Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity
Samuel Horváth
Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learn…
FedShuffle: Recipes for Better Use of Local Work in Federated Learning
Samuel Horváth, Maziar Sanjabi, Lin Xiao +2
The practice of applying several local updates before aggregation across clients has been empirically shown to be a successful approach to overcoming the communication bottleneck i…
FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
Elnur Gasanov, Ahmed Khaled, Samuel Horváth +1
Federated Learning (FL) is an increasingly popular machine learning paradigm in which multiple nodes try to collaboratively learn under privacy, communication and multiple heteroge…
Hyperparameter Transfer Learning with Adaptive Complexity
Samuel Horváth, Aaron Klein, Peter Richtárik +1
Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models. In practice, one frequently has to solve similar hyp…
Lower Bounds and Optimal Algorithms for Personalized Federated Learning
Filip Hanzely, Slavomír Hanzely, Samuel Horváth +1
In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richtárik (2020) which was shown to give an alternative…
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning
Samuel Horváth, Peter Richtárik
Modern large-scale machine learning applications require stochastic optimization algorithms to be implemented on distributed compute systems. A key bottleneck of such systems is th…
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization
Samuel Horváth, Lihua Lei, Peter Richtárik +1
Adaptivity is an important yet under-studied property in modern optimization theory. The gap between the state-of-the-art theory and the current practice is striking in that algori…
Stochastic Distributed Learning with Gradient Quantization and Variance Reduction
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko +2
We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate the co…