2 citations · 3 across the 15 of their papers we have counts for
6 papers · 1 filter
D-Quant: Driftable Entropy Coding for KV Cache Quantization
Yi Su, Hong Liu, Guanghua Yu +1
The KV cache has become a major bottleneck in deploying LLMs, as its memory footprint grows linearly with sequence length and batch size, imposing substantial pressure on both memo…
Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
Li Wang, Yi Su, Xiabao Wu +9
Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memo…
PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
Hong Liu, Yuan Cheng, Lin Niu +5
Token-level sparse attention, as implemented by DeepSeek Sparse Attention (DSA) in production systems, makes the downstream attention efficient but shifts the bottleneck to the ind…
LongFlow: Efficient KV Cache Compression for Reasoning Models
Yi Su, Zhenxu Tian, Dan Qiao +3
Recent reasoning models such as OpenAI-o1 and DeepSeek-R1 have shown strong performance on complex tasks including mathematical reasoning and code generation. However, this perform…
Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
Zhenxu Tian, Yi Su, Juntao Li +1
The Key-Value (KV) cache is crucial for efficient Large Language Models (LLMs) inference, but excessively long contexts drastically increase KV cache memory footprint. Existing KV…
Quantized Inference for OneRec-V2
Yi Su, Xinchen Luo, Hongtao Cheng +7
Quantized inference has demonstrated substantial system-level benefits in large language models while preserving model quality. In contrast, reliably applying low-precision quantiz…