133 citations · 170 across the 5 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024★ 6 cited
Investigating Data Contamination for Pre-training Language Models
Minhao Jiang, Ken Ziyu Liu, Ming Zhong +4
Language models pre-trained on web-scale corpora demonstrate impressive capabilities on diverse downstream tasks. However, there is increasing concern whether such capabilities mig…
cs.CL2023★ 4 cited
Pretraining on the Test Set Is All You Need
Rylan Schaeffer
Inspired by recent work demonstrating the promise of smaller Transformer-based language models pretrained on carefully curated data, we supercharge such approaches by investing hea…