2.1k citations · 4.2k across the 25 of their papers we have counts for
Showing 2020 · cs.CLShow all
2 papers · 2 filters
cs.CL2020★ 351 cited
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu +6
Neural network scaling has been critical for improving the model quality in many real-world machine learning applications with vast amounts of training data and compute. Although t…
cs.CL2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model?
Adam Roberts, Colin Raffel, Noam Shazeer
It has recently been observed that neural language models trained on unstructured text can implicitly store and retrieve knowledge using natural language queries. In this short pap…