69 citations · 300 across the 38 of their papers we have counts for
41 papers
MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs
Yuhang Zhou, Giannis Karamanolakis, Victor Soto +5
The recent success of specialized Large Language Models (LLMs) in domains such as mathematical reasoning and coding has led to growing interest in methods for merging these expert…
Deconstructing In-Context Learning: Understanding Prompts via Corruption
Namrata Shivagunde, Vladislav Lialin, Sherin Muckatira +1
The ability of large language models (LLMs) to learn in context based on the provided prompt has led to an explosive growth in their use, culminating in the proliferation of…
Emergent Abilities in Reduced-Scale Generative Language Models
Sherin Muckatira, Vijeta Deshpande, Vladislav Lialin +1
Large language models can solve new tasks without task-specific fine-tuning. This ability, also known as in-context learning (ICL), is considered an emergent ability and is primari…
Prompt Perturbation Consistency Learning for Robust Language Models
Yao Qiang, Subhrangshu Nandi, Ninareh Mehrabi +4
Large language models (LLMs) have demonstrated impressive performance on a number of natural language processing tasks, such as question answering and text summarization. However,…
Let's Reinforce Step by Step
Sarah Pan, Vladislav Lialin, Sherin Muckatira +1
While recent advances have boosted LM proficiency in linguistic benchmarks, LMs consistently struggle to reason correctly on complex tasks like mathematics. We turn to Reinforcemen…
ReLoRA: High-Rank Training Through Low-Rank Updates
Vladislav Lialin, Namrata Shivagunde, Sherin Muckatira +1
Despite the dominance and effectiveness of scaling, resulting in large networks with hundreds of billions of parameters, the necessity to train overparameterized models remains poo…