Transferable text data distillation by trajectory matching

Rong Yao, Hailin Hu, Yifei Fu +5

In the realm of large language model (LLM), as the size of large models increases, it also brings higher training costs. There is a urgent need to minimize the data size in LLM tra…