1 citations · 1 across the 11 of their papers we have counts for
12 papers
Stable-Plastic Subspace Construction for Differentially Private Continual LoRA
Jin Liu, Ning Xi, Yinbin Miao +3
Continual learning (CL) adapts pretrained models to a sequence of tasks while retaining knowledge acquired from earlier tasks. When task data are sensitive, differential privacy (D…
When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy
Jin Liu, Junkang Liu, Ning Xi +4
Model merging promises to construct a single multi-task model from independently fine-tuned task models without accessing the original task data. This makes it attractive when task…
FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
Junkang Liu, Fanhua Shang, Yuanyuan Liu +3
Although Federated Learning has been widely studied in recent years, there are still high overhead expenses in each communication round for large-scale models such as Vision Transf…
FedNSAM:Consistency of Local and Global Flatness for Federated Learning
Junkang Liu, Fanhua Shang, Yuxuan Tian +2
In federated learning (FL), multi-step local updates and data heterogeneity usually lead to sharper global minima, which degrades the performance of the global model. Popular FL al…
DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models
Jin Liu, Yinbin Miao, Ning Xi +1
Balancing convergence efficiency and robustness under Differential Privacy (DP) is a central challenge in Federated Learning (FL). While AdamW accelerates training and fine-tuning…
Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models
Jin Liu, Yinbin Miao, Ning Xi +1
Fine-tuning large vision models (LVMs) and large language models (LLMs) under differentially private federated learning (DPFL) is hindered by a fundamental privacy-utility trade-of…
Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data
Junkang Liu, Fanhua Shang, Hongying Liu +3
Second-order optimizers can significantly accelerate large-scale training, yet their naive federated variants are often unstable or even diverge on non-IID data. We show that a key…
ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation
Junchao Zhou, Junkang Liu, Fanhua Shang
Federated Learning with Low-Rank Adaptation (LoRA) faces three critical challenges under client heterogeneity: (1) Initialization-Induced Instability due to random initialization m…
DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm
Junkang Liu, Yuxuan Tian, Fanhua Shang +4
To prevent inference attacks in Federated Learning (FL) and reduce the leakage of sensitive information, Client-level Differentially Private Federated Learning (CL-DPFL) is widely…
FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
Junkang Liu, Fanhua Shang, Junchao Zhou +3
The core bottleneck of Federated Learning (FL) lies in the communication rounds. That is, how to achieve more effective local updates is crucial for reducing communication rounds.…
FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models
Junkang Liu, Fanhua Shang, Hongying Liu +5
AdamW has become one of the most effective optimizers for training large-scale models. We have also observed its effectiveness in the context of federated learning (FL). However, d…
FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging
Liu junkang, Yuanyuan Liu, Fanhua Shang +3
For federated learning (FL) algorithms such as FedSAM, their generalization capability is crucial for real-word applications. In this paper, we revisit the generalization problem i…