2.2k citations · 2.5k across the 19 of their papers we have counts for
Showing 2024 · cs.CLShow all
2 papers · 2 filters
cs.CL2024★ 5 cited
Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models
Jupinder Parmar, Sanjev Satheesh, Mostofa Patwary +2
As language models have scaled both their number of parameters and pretraining dataset sizes, the computational cost for pretraining has become intractable except for the most well…
cs.CL2024★ 7 cited
Nemotron-4 340B Technical Report
Nvidia, :, Bo Adler +80
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open…