Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores

Chenpeng Wu, Qiqi Gu, Heng Shi +2

The escalating size of Mixture-of-Experts (MoE) based Large Language Models (LLMs) presents significant computational and memory challenges, necessitating innovative solutions to e…