Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference
Zhenxu Tian, Kebin Liu, Zhengwu Yang +8
Long-context decoding in large language models (LLMs) is increasingly bottlenecked by the memory and computation required to attend over an ever-growing Key-Value (KV) cache at eac…
cs.CL2026
Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
Zhenxu Tian, Yi Su, Juntao Li +1
The Key-Value (KV) cache is crucial for efficient Large Language Models (LLMs) inference, but excessively long contexts drastically increase KV cache memory footprint. Existing KV…
cs.CL2025
Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification
Jikai Wang, Zhenxu Tian, Juntao Li +5
Recent works have revealed the great potential of speculative decoding in accelerating the autoregressive generation process of large language models. The success of these methods…