6 citations · 14 across the 17 of their papers we have counts for
3 papers · 1 filter
Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
Feiyang Kang, Newsha Ardalani, Michael Kuchnik +7
Training data plays a crucial role in Large Language Models (LLM) scaling, yet high quality data is of limited supply. Synthetic data techniques offer a potential path toward sides…
Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
Feiyang Kang, Michael Kuchnik, Karthik Padthe +4
In post-training for reasoning Large Language Models (LLMs), the current state of practice trains LLMs in two independent stages: Supervised Fine-Tuning (SFT) and Reinforcement Lea…
AdaDeDup: Adaptive Hybrid Data Pruning for Efficient Large-Scale Object Detection Training
Feiyang Kang, Nadine Chang, Maying Shen +4
The computational burden and inherent redundancy of large-scale datasets challenge the training of contemporary machine learning models. Data pruning offers a solution by selecting…