1 citations · 1 across the 12 of their papers we have counts for
Showing 2024 · cs.CLShow all
2 papers · 2 filters
cs.CL2024
Training Bilingual LMs with Data Constraints in the Targeted Language
Skyler Seto, Maartje ter Hoeve, Richard He Bai +2
Large language models are trained on massive scrapes of the web, as required by current scaling laws. Most progress is made for English, given its abundance of high-quality pretrai…
cs.CL2024
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
David Grangier, Simin Fan, Skyler Seto +1
Specialist language models (LMs) focus on a specific task or domain on which they often outperform generalist LMs of the same size. However, the specialist data needed to pretrain…