activity
20242026
most citedSmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs

1 citations · 1 across the 4 of their papers we have counts for

collaborators
Showing 2026Show all

1 paper · 1 filter

cs.CL2026

Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets

Sultan Alrashed, Francesco Orabona

Multilingual data from the web is essential for LLM pretraining. Yet, scraping it is expensive, and research groups repeatedly crawl the same content. For example, we found that ov…