133 citations · 281 across the 44 of their papers we have counts for
Showing 2023 · cs.CLShow all
3 papers · 2 filters
cs.CL2023★ 4 cited
Pretraining on the Test Set Is All You Need
Rylan Schaeffer
Inspired by recent work demonstrating the promise of smaller Transformer-based language models pretrained on carefully curated data, we supercharge such approaches by investing hea…
cs.CL2023★ 6 cited
Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
Brando Miranda, Alycia Lee, Sudharsan Sundar +4
Current trends in pre-training Large Language Models (LLMs) primarily focus on the scaling of model and dataset size. While the quality of pre-training data is considered an import…
cs.CL2023★ 62 cited
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Boxin Wang, Weixin Chen, Hengzhi Pei +16
Generative Pre-trained Transformer (GPT) models have exhibited exciting progress in their capabilities, capturing the interest of practitioners and the public alike. Yet, while the…