4 papers
ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models
Yujia Wang, Yuanpu Cao, Jinghui Chen
Federated learning (FL) has been extensively studied as a privacy-preserving training paradigm. Recently, federated block coordinate descent scheme has become a popular option in t…
Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning
Yujia Wang, Fenglong Ma, Jinghui Chen
Asynchronous federated learning (FL) has recently gained attention for its enhanced efficiency and scalability, enabling local clients to send model updates to the server at their…
AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections
Xin Yu, Yujia Wang, Jinghui Chen +1
Low-Rank Adaptation (LoRA) has emerged as an effective technique for reducing memory overhead in fine-tuning large language models. However, it often suffers from sub-optimal perfo…
JoPA:Explaining Large Language Model's Generation via Joint Prompt Attribution
Yurui Chang, Bochuan Cao, Yujia Wang +2
Large Language Models (LLMs) have demonstrated impressive performances in complex text generation tasks. However, the contribution of the input prompt to the generated content stil…