Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference

Haolin Zhang, Jeff Huang

The common assumption in on-device AI is that GPUs, with their superior parallel processing, always provide the best performance for large language model (LLM) inference. In this w…