SMEPilot: Characterizing and Optimizing LLM Inference with Scalable Matrix Extensions

Feiyang Chen, Haibo Chen

Modern CPUs increasingly integrate matrix extensions, such as Arm Scalable Matrix Extension (SME), that provide high-throughput matrix execution within the CPU. For LLM inference,…