A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources

Qingyu Song, Rui Liu, Wei Lin +11

Deploying Large Language Models (LLMs) on edge devices enhances privacy but faces performance hurdles due to limited resources. We introduce a systematic methodology to evaluate on…