6 papers
Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach
Xinyi Xu, Bingnan Xiao, Shuang Qin +2
Low-rank adaptation (LoRA) represents large language model (LLM) updates with two compact matrix factors, i.e., and , providing an efficient way to fine-tune large models in…
Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks
Nuocheng Yang, Sihua Wang, Zihan Chen +2
Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and se…
FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
Bingnan Xiao, Yuan Gao, Bingcong Li +3
Sharpness-aware minimization (SAM) is an effective method for improving the generalization of federated learning (FL) by steering local training toward flat minima. Under data hete…
Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
Nuocheng Yang, Sihua Wang, Ouwen Huan +3
Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM)…
Robust Federated Fine-Tuning in Heterogeneous Networks with Unreliable Connections: An Aggregation View
Yanmeng Wang, Zhiwen Dai, Shuai Wang +4
Federated Fine-Tuning (FFT) has attracted growing interest as it leverages both server- and client-side data to enhance global model generalization while preserving privacy, and si…
Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks
Shuai Wang, Yanqing Xu, Chaoqun You +2
Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hin…