5 papers
Skill-Targeted Adaptive Training
Yinghui He, Abhishek Panigrahi, Yong Lin +1
Language models often show little to no improvement (i.e., "saturation") when trained via vanilla supervised fine-tuning (SFT) on data similar to what they saw in their training se…
AdaptMI: Adaptive Skill-based In-context Math Instruction for Small Language Models
Yinghui He, Abhishek Panigrahi, Yong Lin +1
In-context learning (ICL) allows a language model to improve its problem-solving capability when provided with suitable information in context. Since the choice of in-context infor…
On the Power of Context-Enhanced Learning in LLMs
Xingyu Zhu, Abhishek Panigrahi, Sanjeev Arora
We formalize a new concept for LLMs, context-enhanced learning. It involves standard gradient-based learning on text except that the context is enhanced with additional data on whi…
Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?
Simon Park, Abhishek Panigrahi, Yun Cheng +3
Vision Language Models (VLMs) are impressive at visual question answering and image captioning. But they underperform on multi-step visual reasoning -- even compared to LLMs on the…
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
Sadhika Malladi, Kaifeng Lyu, Abhishek Panigrahi +1
Approximating Stochastic Gradient Descent (SGD) as a Stochastic Differential Equation (SDE) has allowed researchers to enjoy the benefits of studying a continuous optimization traj…