Fast On-device LLM Inference with NPUs

Daliang Xu, Hao Zhang, Liming Yang +4

On-device inference for Large Language Models (LLMs), driven by increasing privacy concerns and advancements of mobile-sized models, has gained significant interest. However, even…