Diversity Measurement and Subset Selection for Instruction Tuning Datasets

Peiqi Wang, Yikang Shen, Zhen Guo +4

We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset…