Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training
Powei Chang, Jinpeng Zhang, Bowen Chen +9
Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling gre…
cs.LG2024
Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
Shuhe Wang, Guoyin Wang, Yizhong Wang +3
Packing, initially utilized in the pre-training phase, is an optimization technique designed to maximize hardware resource efficiency by combining different training sequences to f…