Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
Zihao An, Taichi Liu, Ziqiong Liu +3
Speculative decoding accelerates Large Language Models (LLMs) inference by using a lightweight draft model to propose candidate tokens that are verified in parallel by the target m…
cs.CL2026
NeuronMLP: Efficient LLM Inference via Singular Value Decomposition Compression and Tiling on AWS Trainium
Dinghong Song, Jierui Xu, Weichu Yang +2
Emerging AI accelerators have started to gain attention and offer new opportunities for efficient inference of large language models (LLMs). Trainium, an AI accelerator recently de…
cs.CL2025
AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
Dinghong Song, Yuan Feng, Yiwei Wang +6
Large Language Models (LLMs) are widely used in generative applications such as chatting, code generation, and reasoning. However, many realworld workloads such as classification,…