4 papers
Position: Let's Develop Data Probes to Fundamentally Understand How Data Affects LLM Performance
Shiqiang Wang, Herbert Woisetschläger, Hans Arno Jacobsen +1
Data is fundamental to large language models (LLMs). However, understanding of what makes certain data useful for different stages of an LLM workflow, including training, tuning, a…
Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization
Jake B. Perazzone, Shiqiang Wang, Mingyue Ji +1
Federated learning (FL) is a useful tool that enables the training of machine learning models over distributed data without having to collect data centrally. When deploying FL in c…
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization
Jiayi Wang, Shiqiang Wang, Rong-Rong Chen +1
In federated learning (FL), data heterogeneity is the main reason that existing theoretical analyses are pessimistic about the convergence rate. In particular, for many FL algorith…
A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging
Shiqiang Wang, Mingyue Ji
In federated learning (FL), clients usually have diverse participation statistics that are unknown a priori, which can significantly harm the performance of FL if not handled prope…