1.6k citations · 3.9k across the 52 of their papers we have counts for
Showing 2024 · cs.CLShow all
2 papers · 2 filters
cs.CL2024★ 1 cited
On the Inductive Bias of Stacking Towards Improving Reasoning
Nikunj Saunshi, Stefani Karp, Shankar Krishnan +3
Given the increasing scale of model sizes, novel training strategies like gradual stacking [Gong et al., 2019, Reddi et al., 2023] have garnered interest. Stacking enables efficien…
cs.CL2024
Efficient Stagewise Pretraining via Progressive Subnetworks
Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu +4
Recent developments in large language models have sparked interest in efficient pretraining methods. Stagewise training approaches to improve efficiency, like gradual stacking and…