ModuleFormer: Modularity Emerges from Mixture-of-Experts

Yikang Shen, Zheyu Zhang, Tianyou Cao +3

Large Language Models (LLMs) have achieved remarkable results. However, existing models are expensive to train and deploy, and it is also difficult to expand their knowledge beyond…