69 citations · 90 across the 11 of their papers we have counts for
6 papers · 1 filter
Let's Reinforce Step by Step
Sarah Pan, Vladislav Lialin, Sherin Muckatira +1
While recent advances have boosted LM proficiency in linguistic benchmarks, LMs consistently struggle to reason correctly on complex tasks like mathematics. We turn to Reinforcemen…
ReLoRA: High-Rank Training Through Low-Rank Updates
Vladislav Lialin, Namrata Shivagunde, Sherin Muckatira +1
Despite the dominance and effectiveness of scaling, resulting in large networks with hundreds of billions of parameters, the necessity to train overparameterized models remains poo…
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale
Vijeta Deshpande, Dan Pechi, Shree Thatte +2
In recent years, language models have drastically grown in size, and the abilities of these models have been shown to improve with scale. The majority of recent scaling laws studie…
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data
Vladislav Lialin, Stephen Rawls, David Chan +3
Scaling up weakly-supervised datasets has shown to be highly effective in the image-text domain and has contributed to most of the recent state-of-the-art computer vision and multi…
Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning
Namrata Shivagunde, Vladislav Lialin, Anna Rumshisky
Language model probing is often used to test specific capabilities of models. However, conclusions from such studies may be limited when the probing benchmarks are small and lack s…
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning
Vladislav Lialin, Vijeta Deshpande, Xiaowei Yao +1
This paper presents a systematic overview of parameter-efficient fine-tuning methods, covering over 50 papers published between early 2019 and mid-2024. These methods aim to addres…