3 citations · 9 across the 8 of their papers we have counts for
6 papers · 1 filter
Theory on Attention Dynamics for Out-of-Distribution In-Context Learning
Junze Deng, Daouda Sow, Sen Lin +1
Transformers have demonstrated remarkable in-context learning (ICL) capabilities, enabling them to perform new tasks without additional fine-tuning. However, their performance ofte…
Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
Daouda Sow, Herbert Woisetschläger, Saikiran Bulusu +3
Pretraining large language models (LLMs) on vast and heterogeneous datasets is crucial for achieving state-of-the-art performance across diverse downstream tasks. However, current…
Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization
Xuxi Chen, Zhendong Wang, Daouda Sow +5
In the rapidly advancing arena of large language models (LLMs), a key challenge is to enhance their capabilities amid a looming shortage of high-quality training data. Our study st…
Doubly Robust Instance-Reweighted Adversarial Training
Daouda Sow, Sen Lin, Zhangyang Wang +1
Assigning importance weights to adversarial data has achieved great success in training adversarially robust networks under limited model capacity. However, existing instance-rewei…
Algorithm Design for Online Meta-Learning with Task Boundary Detection
Daouda Sow, Sen Lin, Yingbin Liang +1
Online meta-learning has recently emerged as a marriage between batch meta-learning and online learning, for achieving the capability of quick adaptation on new tasks in a lifelong…
On the Convergence Theory for Hessian-Free Bilevel Algorithms
Daouda Sow, Kaiyi Ji, Yingbin Liang
Bilevel optimization has arisen as a powerful tool in modern machine learning. However, due to the nested structure of bilevel optimization, even gradient-based methods require sec…