832 citations · 1.4k across the 48 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 10 cited
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
Deepak Narayanan, Mohammad Shoeybi, Jared Casper +9
Large language models have led to state-of-the-art accuracies across a range of tasks. However, training these models efficiently is challenging for two reasons: a) GPU memory capa…
cs.CL2021★ 1 cited
End-to-End Training of Neural Retrievers for Open-Domain Question Answering
Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi +4
Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised approaches. However, it remains unclear how unsu…