A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration

Wei-Hsing Huang, Janak Sharda, Cheng-Jhih Shih +6

Conventional large language models (LLMs) are equipped with dozens of GB to TB of model parameters, making inference highly energy-intensive and costly as all the weights need to b…