242 citations · 647 across the 20 of their papers we have counts for
22 papers
Personalized Federated Learning with Multiple Known Clusters
Boxiang Lyu, Filip Hanzely, Mladen Kolar
We consider the problem of personalized federated learning when there are known cluster structures within users. An intuitive approach would be to regularize the parameters so that…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimization
Mher Safaryan, Filip Hanzely, Peter Richtárik
Large scale distributed optimization has become the default tool for the training of supervised machine learning models with a large number of parameters and training data. Recent…
Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
Filip Hanzely, Boxin Zhao, Mladen Kolar
We investigate the optimization aspects of personalized Federated Learning (FL). We propose general optimizers that can be applied to numerous existing personalized FL objectives,…
Local SGD: Unified Theory and New Efficient Methods
Eduard Gorbunov, Filip Hanzely, Peter Richtárik
We present a unified framework for analyzing local SGD methods in the convex and strongly convex regimes for distributed/federated training of supervised machine learning models. W…
Lower Bounds and Optimal Algorithms for Personalized Federated Learning
Filip Hanzely, Slavomír Hanzely, Samuel Horváth +1
In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richtárik (2020) which was shown to give an alternative…
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters
Filip Hanzely
Many key problems in machine learning and data science are routinely modeled as optimization problems and solved via optimization algorithms. With the increase of the volume of dat…
Stochastic Subspace Cubic Newton Method
Filip Hanzely, Nikita Doikov, Peter Richtárik +1
In this paper, we propose a new randomized second-order optimization algorithm---Stochastic Subspace Cubic Newton (SSCN)---for minimizing a high dimensional convex function . Ou…
Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems
Filip Hanzely, Dmitry Kovalev, Peter Richtarik
We propose an accelerated version of stochastic variance reduced coordinate descent -- ASVRCD. As other variance reduced coordinate descent methods such as SEGA or SVRCD, our metho…
Federated Learning of a Mixture of Global and Local Models
Filip Hanzely, Peter Richtárik
We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to fin…
Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit
Aritra Dutta, Filip Hanzely, Jingwei Liang +1
The best pair problem aims to find a pair of points that minimize the distance between two disjoint sets. In this paper, we formulate the classical robust principal component analy…
A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent
Eduard Gorbunov, Filip Hanzely, Peter Richtárik
In this paper we introduce a unified analysis of a large family of variants of proximal stochastic gradient descent ({\tt SGD}) which so far have required different intuitions, con…
One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methods
Filip Hanzely, Peter Richtárik
We propose a remarkably general variance-reduced method suitable for solving regularized empirical risk minimization problems with either a large number of training examples, or a…
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
Filip Hanzely, Jakub Konečný, Nicolas Loizou +2
In this work we present a randomized gossip algorithm for solving the average consensus problem while at the same time protecting the information about the initial private values s…
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it
Konstantin Mishchenko, Filip Hanzely, Peter Richtárik
Many popular distributed optimization methods for training machine learning models fit the following template: a local gradient estimate is computed independently by each worker, t…
SEGA: Variance Reduction via Gradient Sketching
Filip Hanzely, Konstantin Mishchenko, Peter Richtarik
We propose a randomized first order optimization method--SEGA (SkEtched GrAdient method)-- which progressively throughout its iterations builds a variance-reduced estimate of the g…
Accelerated Coordinate Descent with Arbitrary Sampling and Best Rates for Minibatches
Filip Hanzely, Peter Richtárik
Accelerated coordinate descent is a widely popular optimization algorithm due to its efficiency on large-dimensional problems. It achieves state-of-the-art complexity on an importa…
Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization
Filip Hanzely, Peter Richtarik, Lin Xiao
We consider the problem of minimizing the sum of two convex functions: one is differentiable and relatively smooth with respect to a reference convex function, and the other can be…
A Nonconvex Projection Method for Robust PCA
Aritra Dutta, Filip Hanzely, Peter Richtárik
Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components. In this paper, we propose…
Fastest Rates for Stochastic Mirror Descent Methods
Filip Hanzely, Peter Richtárik
Relative smoothness - a notion introduced by Birnbaum et al. (2011) and rediscovered by Bauschke et al. (2016) and Lu et al. (2016) - generalizes the standard notion of smoothness…
Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization
Robert M. Gower, Filip Hanzely, Peter Richtárik +1
We present the first accelerated randomized algorithm for solving linear systems in Euclidean spaces. One essential problem of this type is the matrix inversion problem. In particu…
Privacy Preserving Randomized Gossip Algorithms
Filip Hanzely, Jakub Konečný, Nicolas Loizou +2
In this work we present three different randomized gossip algorithms for solving the average consensus problem while at the same time protecting the information about the initial p…