Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency
Hanyu Zhao, Li Du, Yiming Ju +2
With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs).…
cs.CL2024
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
Bo-Wen Zhang, Liangdong Wang, Ye Yuan +24
In recent years, with the rapid application of large language models across various fields, the scale of these models has gradually increased, and the resources required for their…