162 citations · 423 across the 11 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2021★ 3 cited
Speeding up Deep Model Training by Sharing Weights and Then Unsharing
Shuo Yang, Le Hou, Xiaodan Song +2
We propose a simple and efficient approach for training the BERT model. Our approach exploits the special structure of BERT that contains a stack of repeated modules (i.e., transfo…
cs.LG2020★ 5 cited
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
Denny Zhou, Mao Ye, Chen Chen +6
For deploying a deep learning model into production, it needs to be both accurate and compact to meet the latency and memory constraints. This usually results in a network that is…
cs.LG2019★ 162 cited
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
Yang You, Jing Li, Sashank Reddi +7
Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization me…