3 citations · 9 across the 8 of their papers we have counts for
9 papers
Theory on Attention Dynamics for Out-of-Distribution In-Context Learning
Junze Deng, Daouda Sow, Sen Lin +1
Transformers have demonstrated remarkable in-context learning (ICL) capabilities, enabling them to perform new tasks without additional fine-tuning. However, their performance ofte…
Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
Daouda Sow, Herbert Woisetschläger, Saikiran Bulusu +3
Pretraining large language models (LLMs) on vast and heterogeneous datasets is crucial for achieving state-of-the-art performance across diverse downstream tasks. However, current…
Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization
Xuxi Chen, Zhendong Wang, Daouda Sow +5
In the rapidly advancing arena of large language models (LLMs), a key challenge is to enhance their capabilities amid a looming shortage of high-quality training data. Our study st…
Non-Convex Bilevel Optimization with Time-Varying Objective Functions
Sen Lin, Daouda Sow, Kaiyi Ji +2
Bilevel optimization has become a powerful tool in a wide variety of machine learning problems. However, the current nonconvex bilevel optimization considers an offline dataset and…
Doubly Robust Instance-Reweighted Adversarial Training
Daouda Sow, Sen Lin, Zhangyang Wang +1
Assigning importance weights to adversarial data has achieved great success in training adversarially robust networks under limited model capacity. However, existing instance-rewei…
Algorithm Design for Online Meta-Learning with Task Boundary Detection
Daouda Sow, Sen Lin, Yingbin Liang +1
Online meta-learning has recently emerged as a marriage between batch meta-learning and online learning, for achieving the capability of quick adaptation on new tasks in a lifelong…
A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima
Daouda Sow, Kaiyi Ji, Ziwei Guan +1
Bilevel optimization has found extensive applications in modern machine learning problems such as hyperparameter optimization, neural architecture search, meta-learning, etc. While…
On the Convergence Theory for Hessian-Free Bilevel Algorithms
Daouda Sow, Kaiyi Ji, Yingbin Liang
Bilevel optimization has arisen as a powerful tool in modern machine learning. However, due to the nested structure of bilevel optimization, even gradient-based methods require sec…
A sequential guiding network with attention for image captioning
Daouda Sow, Zengchang Qin, Mouhamed Niasse +1
The recent advances of deep learning in both computer vision (CV) and natural language processing (NLP) provide us a new way of understanding semantics, by which we can deal with m…