most citedSkywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

4 citations · 4 across the 1 of their papers we have counts for

collaborators

1 paper

cs.CL2024★ 4 cited

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

Tianwen Wei, Bo Zhu, Liang Zhao +13

In this technical report, we introduce the training methodologies implemented in the development of Skywork-MoE, a high-performance mixture-of-experts (MoE) large language model (L…