From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning

Haoyu Li, Xuhong Li, Yiming Dong +1

Dataset diversity plays a pivotal role for the successful training of many machine learning models, particularly in the supervised fine-tuning (SFT) stage of large language model (…