Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference

Haiyang Huang, Newsha Ardalani, Anna Sun +6

Mixture-of-Experts (MoE) models have gained popularity in achieving state-of-the-art performance in a wide range of tasks in computer vision and natural language processing. They e…