40 citations · 223 across the 28 of their papers we have counts for
19 papers · 1 filter
FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
Junkang Liu, Fanhua Shang, Yuanyuan Liu +3
Although Federated Learning has been widely studied in recent years, there are still high overhead expenses in each communication round for large-scale models such as Vision Transf…
FedNSAM:Consistency of Local and Global Flatness for Federated Learning
Junkang Liu, Fanhua Shang, Yuxuan Tian +2
In federated learning (FL), multi-step local updates and data heterogeneity usually lead to sharper global minima, which degrades the performance of the global model. Popular FL al…
Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data
Junkang Liu, Fanhua Shang, Hongying Liu +3
Second-order optimizers can significantly accelerate large-scale training, yet their naive federated variants are often unstable or even diverge on non-IID data. We show that a key…
DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm
Junkang Liu, Yuxuan Tian, Fanhua Shang +4
To prevent inference attacks in Federated Learning (FL) and reduce the leakage of sensitive information, Client-level Differentially Private Federated Learning (CL-DPFL) is widely…
FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
Junkang Liu, Fanhua Shang, Junchao Zhou +3
The core bottleneck of Federated Learning (FL) lies in the communication rounds. That is, how to achieve more effective local updates is crucial for reducing communication rounds.…
FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models
Junkang Liu, Fanhua Shang, Hongying Liu +5
AdamW has become one of the most effective optimizers for training large-scale models. We have also observed its effectiveness in the context of federated learning (FL). However, d…
FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging
Liu junkang, Yuanyuan Liu, Fanhua Shang +3
For federated learning (FL) algorithms such as FedSAM, their generalization capability is crucial for real-word applications. In this paper, we revisit the generalization problem i…
Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning
Hua Huang, Fanhua Shang, Yuanyuan Liu +1
Federated Learning (FL) has become an active and promising distributed machine learning paradigm. As a result of statistical heterogeneity, recent studies clearly show that the per…
Learned Interpretable Residual Extragradient ISTA for Sparse Coding
Lin Kong, Wei Sun, Fanhua Shang +2
Recently, the study on learned iterative shrinkage thresholding algorithm (LISTA) has attracted increasing attentions. A large number of experiments as well as some theories have p…
Differentially Private ADMM Algorithms for Machine Learning
Tao Xu, Fanhua Shang, Yuanyuan Liu +3
In this paper, we study efficient differentially private alternating direction methods of multipliers (ADMM) via gradient perturbation for many machine learning problems. For smoot…
Boosting Gradient for White-Box Adversarial Attacks
Hongying Liu, Zhenyu Zhou, Fanhua Shang +3
Deep neural networks (DNNs) are playing key roles in various artificial intelligence applications such as image classification and object recognition. However, a growing number of…
Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization
Fanhua Shang, Bingkun Wei, Hongying Liu +2
Large-scale non-convex sparsity-constrained problems have recently gained extensive attention. Most existing deterministic optimization methods (e.g., GraSP) are not suitable for l…
Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications
Fanhua Shang, James Cheng, Yuanyuan Liu +2
The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background m…
Tractable and Scalable Schatten Quasi-Norm Approximations for Rank Minimization
Fanhua Shang, Yuanyuan Liu, James Cheng
The Schatten quasi-norm was introduced to bridge the gap between the trace norm and rank function. However, existing algorithms are too slow or even impractical for large-scale pro…
Accelerated Variance Reduced Stochastic ADMM
Yuanyuan Liu, Fanhua Shang, James Cheng
Recently, many variance reduced stochastic alternating direction method of multipliers (ADMM) methods (e.g.\ SAG-ADMM, SDCA-ADMM and SVRG-ADMM) have made exciting progress such as…
Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning
Fanhua Shang, Yuanyuan Liu, James Cheng +1
Recently, research on accelerated stochastic gradient descent methods (e.g., SVRG) has made exciting progress (e.g., linear convergence for strongly convex problems). However, the…
Guaranteed Sufficient Decrease for Variance Reduced Stochastic Gradient Descent
Fanhua Shang, Yuanyuan Liu, James Cheng +2
In this paper, we propose a novel sufficient decrease technique for variance reduced stochastic gradient descent methods such as SAG, SVRG and SAGA. In order to make sufficient dec…
Regularized Orthogonal Tensor Decompositions for Multi-Relational Learning
Fanhua Shang, James Cheng, Hong Cheng
Multi-relational learning has received lots of attention from researchers in various research communities. Most existing methods either suffer from superlinear per-iteration cost,…
Structured Low-Rank Matrix Factorization with Missing and Grossly Corrupted Observations
Fanhua Shang, Yuanyuan Liu, Hanghang Tong +2
Recovering low-rank and sparse matrices from incomplete or corrupted observations is an important problem in machine learning, statistics, bioinformatics, computer vision, as well…