Aligning Instruction Tuning with Pre-training

Yiming Liang, Tianyu Zheng, Xinrun Du +12

Instruction tuning enhances large language models (LLMs) to follow human instructions across diverse tasks, relying on high-quality datasets to guide behavior. However, these datas…