4 citations · 7 across the 2 of their papers we have counts for
2 papers
cs.CL2024★ 4 cited
MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining
Jacob Portes, Alex Trott, Sam Havens +6
Although BERT-style encoder models are heavily used in NLP research, many researchers do not pretrain their own BERTs from scratch due to the high cost of training. In the past hal…
cs.LG2024★ 3 cited
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Nikhil Sardana, Jacob Portes, Sasha Doubov +1
Large language model (LLM) scaling laws are empirical formulas that estimate changes in model quality as a result of increasing parameter count and training data. However, these fo…