lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models

Haoxin Wang, Xiaolong Tu, Hongyu Ke +3

Large Language Models (LLMs) are increasingly integrated into everyday applications, but their prevalent cloud-based deployment raises growing concerns around data privacy and long…