Scaling LLM Test-Time Compute with Mobile NPU on Smartphones

Zixu Hao, Jianyu Wei, Tuowei Wang +5

Deploying Large Language Models (LLMs) on mobile devices faces the challenge of insufficient performance in smaller models and excessive resource consumption in larger ones. This p…