2 papers
cs.CL2024
HARE: HumAn pRiors, a key to small language model Efficiency
Lingyun Zhang, Bin jin, Gaojian Ge +7
Human priors play a crucial role in efficiently utilizing data in deep learning. However, with the development of large language models (LLMs), there is an increasing emphasis on s…
cs.CL2023
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Shihan Dou, Enyu Zhou, Yan Liu +13
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.…