BALANCE: Hybrid Autoregressive-Speculative LLM Inference at the Network Edge

Guanqiao Qu, Shuo Chen, Qian Chen +2

Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: A…