4 papers
Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization
Li Shen, Yan Sun, Dacheng Tao
Federated learning (FL) is a distributed paradigm that coordinates massive local clients to collaboratively train a global model via stage-wise local training processes on the hete…
Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs
Yifei Liang, Yan Sun, Xiaochun Cao +1
Push-Sum-based decentralized learning enables optimization over directed communication networks, where information exchange may be asymmetric. While convergence properties of such…
Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning
Ziqing Fan, Yuqiao Xian, Yan Sun +1
A fine-grained data recipe is crucial for pre-training large language models, as it can significantly enhance training efficiency and model performance. One important ingredient in…
A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models
Zihao Lin, Samyadeep Basu, Mohammad Beigi +18
The rise of foundation models has transformed machine learning research, prompting efforts to uncover their inner workings and develop more efficient and reliable applications for…