14 citations · 31 across the 10 of their papers we have counts for
5 papers · 1 filter
GEM-Style Constraints for PEFT with Dual Gradient Projection in LoRA
Brian Tekmen, Jason Yin, Qianqian Tong
Full fine-tuning of Large Language Models (LLMs) is computationally costly, motivating Continual Learning (CL) approaches that utilize parameter-efficient adapters. We revisit Grad…
Federated Nonconvex Sparse Learning
Qianqian Tong, Guannan Liang, Tan Zhu +1
Nonconvex sparse learning plays an essential role in many areas, such as signal processing and deep network compression. Iterative hard thresholding (IHT) methods are the state-of-…
Effective Proximal Methods for Non-convex Non-smooth Regularized Learning
Guannan Liang, Qianqian Tong, Jiahao Ding +2
Sparse learning is a very important tool for mining useful information and patterns from high dimensional data. Non-convex non-smooth regularized learning problems play essential r…
Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data
Qianqian Tong, Guannan Liang, Jinbo Bi
Federated learning allows loads of edge computing devices to collaboratively learn a global model without data sharing. The analysis with partial device participation under non-IID…
Calibrating the Adaptive Learning Rate to Improve Convergence of ADAM
Qianqian Tong, Guannan Liang, Jinbo Bi
Adaptive gradient methods (AGMs) have become popular in optimizing the nonconvex problems in deep learning area. We revisit AGMs and identify that the adaptive learning rate (A-LR)…