Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
Chenkai Xu, Yijie Jin, Jiajun Li +8
Diffusion Large Language Models (dLLMs) have demonstrated significant potential for high-speed inference. However, current confidence-driven decoding strategies are constrained by…
cs.CL2024
In-context KV-Cache Eviction for LLMs via Attention-Gate
Zihao Zeng, Bokai Lin, Tianqi Hou +2
The KV-Cache technique has become the standard for the inference of large language models (LLMs). Yet, it is widely criticized that KV-Cache can become a bottleneck of the LLM infe…