5 papers
FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment
Chuiyang Meng, Ming Tang, Vincent W. S. Wong
Parameter-efficient fine-tuning techniques such as low-rank adaptation (LoRA) enable large language models (LLMs) to adapt to downstream tasks efficiently. Federated learning (FL)…
Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection
Hongyi He, Xiao Liu, Zhenghao Lin +6
High-quality pre-training data is crutial for large language models, where quality captures factual reliability and semantic value, and diversity ensures broad coverage and distrib…
Learning without Global Backpropagation via Synergistic Information Distillation
Chenhao Ye, Ming Tang
Backpropagation (BP), while foundational to deep learning, imposes two critical scalability bottlenecks: update locking, where network modules remain idle until the entire backward…
FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning
Fu Peng, Ming Tang
In federated learning (FL), the data distribution of each client may change over time, introducing both temporal and spatial data heterogeneity, known as concept drift. Data hetero…
An Information-Theoretic Analysis for Federated Learning under Concept Drift
Fu Peng, Meng Zhang, Ming Tang
Recent studies in federated learning (FL) commonly train models on static datasets. However, real-world data often arrives as streams with shifting distributions, causing performan…