66 citations · 324 across the 32 of their papers we have counts for
22 papers · 1 filter
From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learning
Xiangrui Xu, Zhize Li, Yufei Han +3
Data Reconstruction Attacks (DRA) pose a significant threat to Federated Learning (FL) systems by enabling adversaries to infer sensitive training data from local clients. Despite…
Coresets for Clustering Under Stochastic Noise
Lingxiao Huang, Zhize Li, Nisheeth K. Vishnoi +2
We study the problem of constructing coresets for -clustering when the input dataset is corrupted by stochastic noise drawn from a known distribution. In this setting, eval…
X-VFL: A New Vertical Federated Learning Framework with Cross Completion and Decision Subspace Alignment
Qinghua Yao, Xiangrui Xu, Zhize Li
Vertical Federated Learning (VFL) enables collaborative learning by integrating disjoint feature subsets from multiple clients/parties. However, VFL typically faces two key challen…
The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity
Tongle Wu, Zhize Li, Ying Sun
We revisit two fundamental decentralized optimization methods, Decentralized Gradient Tracking (DGT) and Decentralized Gradient Descent (DGD), with multiple local updates. We consi…
Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compression
Sijin Chen, Zhize Li, Yuejie Chi
We consider the problem of finding second-order stationary points of heterogeneous federated learning (FL). Previous works in FL mostly focus on first-order convergence guarantees,…
Coresets for Vertical Federated Learning: Regularized Linear Regression and -Means Clustering
Lingxiao Huang, Zhize Li, Jialin Sun +1
Vertical federated learning (VFL), where data features are stored in multiple parties distributively, is an important area in machine learning. However, the communication complexit…
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization
Zhize Li, Jian Li
We propose and analyze several stochastic gradient algorithms for finding stationary points or local minimum in nonconvex, possibly with nonsmooth regularizer, finite-sum and onlin…
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
Zhize Li, Haoyu Zhao, Boyue Li +1
To enable large-scale machine learning in bandwidth-hungry environments such as wireless networks, significant progress has been made recently in designing communication-efficient…
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation
Peter Richtárik, Igor Sokolov, Ilyas Fatkhullin +3
We propose and study a new class of gradient communication mechanisms for communication-efficient training -- three point compressors (3PC) -- as well as efficient distributed nonc…
BEER: Fast Rate for Decentralized Nonconvex Optimization with Communication Compression
Haoyu Zhao, Boyue Li, Zhize Li +2
Communication efficiency has been widely recognized as the bottleneck for large-scale decentralized machine learning applications in multi-agent or federated environments. To tackl…
Faster Rates for Compressed Federated Learning with Client-Variance Reduction
Haoyu Zhao, Konstantin Burlachenko, Zhize Li +1
Due to the communication bottleneck in distributed and federated learning applications, algorithms using communication compression have attracted significant attention and are wide…
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
Ilyas Fatkhullin, Igor Sokolov, Eduard Gorbunov +2
First proposed by Seide (2014) as a heuristic, error feedback (EF) is a very popular mechanism for enforcing convergence of distributed gradient-based optimization methods enhanced…
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning
Haoyu Zhao, Zhize Li, Peter Richtárik
Federated Averaging (FedAvg, also known as Local-SGD) (McMahan et al., 2017) is a classical federated learning algorithm in which clients run multiple local SGD steps before commun…
CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression
Zhize Li, Peter Richtárik
Due to the high communication cost in distributed and federated learning, methods relying on compressed communication are becoming increasingly popular. Besides, the best theoretic…
ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computation
Zhize Li, Slavomír Hanzely, Peter Richtárik
We propose ZeroSARAH -- a novel variant of the variance-reduced method SARAH (Nguyen et al., 2017) -- for minimizing the average of a large number of nonconvex functions $\frac{1}{…
MARINA: Faster Non-Convex Distributed Learning with Compression
Eduard Gorbunov, Konstantin Burlachenko, Zhize Li +1
We develop and analyze MARINA: a new communication efficient method for non-convex distributed learning over heterogeneous datasets. MARINA employs a novel communication compressio…
PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization
Zhize Li, Hongyan Bao, Xiangliang Zhang +1
In this paper, we propose a novel stochastic gradient estimator -- ProbAbilistic Gradient Estimator (PAGE) -- for nonconvex optimization. PAGE is easy to implement as it is designe…
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization
Rong Ge, Zhize Li, Weiyao Wang +1
Variance reduction techniques like SVRG provide simple and fast algorithms for optimizing a convex finite-sum objective. For nonconvex objectives, these techniques can also find a…
SSRGD: Simple Stochastic Recursive Gradient Descent for Escaping Saddle Points
Zhize Li
We analyze stochastic gradient algorithms for optimizing nonconvex problems. In particular, our goal is to find local minima (second-order stationary points) instead of just findin…
Learning Two-layer Neural Networks with Symmetric Inputs
Rong Ge, Rohith Kuditipudi, Zhize Li +1
We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network $$ y = A σ(Wx) + ξ,…
Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference
Zhize Li, Tianyi Zhang, Shuyu Cheng +2
Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-grad…
Gradient Boosting With Piece-Wise Linear Regression Trees
Yu Shi, Jian Li, Zhize Li
Gradient Boosted Decision Trees (GBDT) is a very successful ensemble learning algorithm widely used across a variety of applications. Recently, several variants of GBDT training al…