4 papers
Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set
Chengwei Wu, Li Du, Hanyu Zhao +4
Scaling the amount of data used for supervied fine-tuning(SFT) does not guarantee the proportional gains in model performance, highlighting a critical need to understand what makes…
CI-VID: A Coherent Interleaved Text-Video Dataset
Yiming Ju, Jijin Hu, Zhengxiong Luo +7
Text-to-video (T2V) generation has recently attracted considerable attention, resulting in the development of numerous high-quality datasets that have propelled progress in this ar…
Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
Jijie Li, Li Du, Hanyu Zhao +5
Large Language Models (LLMs) demonstrate strong performance in real-world applications, yet existing open-source instruction datasets often concentrate on narrow domains, such as m…
Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency
Hanyu Zhao, Li Du, Yiming Ju +2
With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs).…