SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training

Powei Chang, Jinpeng Zhang, Bowen Chen +9

Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling gre…