Showing 2025Show all
2 papers · 1 filter
cs.LG2025
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training
Albert Ge, Tzu-Heng Huang, John Cooper +7
Data mixing strategies have successfully reduced the costs involved in training language models. While promising, such methods suffer from two flaws. First, they rely on predetermi…
cs.LG2025
Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights
Tzu-Heng Huang, Manjot Bilkhu, John Cooper +2
Large-scale web-crawled datasets contain noise, bias, and irrelevant information, necessitating data selection techniques. Existing methods depend on hand-crafted heuristics, downs…