2 papers
cs.AR2026
AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices
Ma Zirui, Fan Zhihua, Li Wenxing +4
Speculative decoding enhances the inference efficiency of large language models (LLMs) by generating drafts using a small draft language model (DLM) and verifying them in batches w…
cs.AR2025
A Systematic Characterization of LLM Inference on GPUs
Haonan Wang, Xuxin Xiao, Mingyu Yan +8
This work presents a systematic characterization of Large Language Model (LLM) inference to address fragmented understanding. Through comprehensive experiments, we establish a four…