4 papers
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
Yanqi Yu, Pingwei Sun, Jianchao Tan +4
Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substantially reducing key/value (KV)…
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
Tao Zhang, Jianchao Tan, Pingwei Sun +5
Softmax attention stores key and value vectors for every preceding token, causing inference memory to grow with sequence length. Recent language models incorporating Gated DeltaNet…
Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning
Mingkang Dong, Hongyi Cai, Xiwen Lei +3
Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for train…
MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
Tao Zhang, Ziqian Zeng, Hao Peng +2
Long Chain-of-Thought (CoT) reasoning has significantly advanced the capabilities of Large Language Models (LLMs), but this progress is accompanied by substantial memory and latenc…