Publications (22)
Federated Learning for Inference at Anytime and Anywhere
Zicheng Liu, Da Li, Javier Fernandez-Marques +6
Federated learning has been predominantly concerned with collaborative training of deep networks from scratch, and especially the many challenges that arise, such as communication…
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
Hao Mark Chen, Shell Xu Hu, Wayne Luk +2
Model merging has emerged as a promising approach for multi-task learning (MTL), offering a data-efficient alternative to conventional fine-tuning. However, with the rapid developm…
CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics
Wanru Zhao, Hongxiang Fan, Shell Xu Hu +3
Recent research has highlighted the importance of data quality in scaling large language models (LLMs). However, automated data quality control faces unique challenges in collabora…
Empirical Bayes Transductive Meta-Learning with Synthetic Gradients
Shell Xu Hu, Pablo G. Moreno, Yang Xiao +4
We propose a meta-learning approach that learns from multiple tasks in a transductive setting, by leveraging the unlabeled query set in addition to the support set to generate a mo…
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
Wanru Zhao, Yihong Chen, Yuzhi Tang +6
Data curation is a critical yet under-explored area in large language model (LLM) training. Existing methods, such as data selection and mixing, operate in an offline paradigm, det…
Strong Baselines for Parameter Efficient Few-Shot Fine-tuning
Samyadeep Basu, Daniela Massiceti, Shell Xu Hu +1
Few-shot classification (FSC) entails learning novel classes given only a few examples per class after a pre-training (or meta-training) phase on a set of base classes. Recent work…