Online Data Selection for Instruction Tuning via Gaussian Processes

Jun Wang, Quoc Phong Nguyen, Julien Monteil +1

With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. E…