DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding

Zewen Jin, Shen Fu, Zeping Duan +8

Mixture-of-Experts (MoE) models have been widely adopted in real-time interactive applications such as coding assistants, real-time audio-video interaction systems. To meet the ext…