Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs

Afsara Benazir, Felix Xiaozhu Lin

Apple Neural Engine (ANE) is a dedicated neural processing unit (NPU) present in every Apple Silicon chip. Mixture-of-Experts (MoE) LLMs improve inference efficiency via sparse act…