dots.llm1 Technical Report

Bi Huo, Bin Tu, Cheng Qin +24

Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this…