11 citations · 15 across the 10 of their papers we have counts for
5 papers · 1 filter
SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
Bailin Wang, Dan Friedman, Tao Lei +1
Block-wise sparse attention offers significant efficiency gains for long-context modeling, yet existing methods often suffer from low selection fidelity and cumulative contextual l…
Synthetic bootstrapped pretraining
Zitong Yang, Aonan Zhang, Hong Liu +4
We introduce Synthetic Bootstrapped Pretraining (SBP), a language model (LM) pretraining procedure that first learns a model of relations between documents from the pretraining dat…
CommVQ: Commutative Vector Quantization for KV Cache Compression
Junyan Li, Yang Zhang, Muhammad Yusuf Hassan +8
Large Language Models (LLMs) are increasingly used in applications requiring long context lengths, but the key-value (KV) cache often becomes a memory bottleneck on GPUs as context…
RATTENTION: Towards the Minimal Sliding Window Size in Local-Global Attention Models
Bailin Wang, Chang Lan, Chong Wang +1
Local-global attention models have recently emerged as compelling alternatives to standard Transformers, promising improvements in both training and inference efficiency. However,…
Recurrent Drafter for Fast Speculative Decoding in Large Language Models
Yunfei Cheng, Aonan Zhang, Xuanyu Zhang +2
We present Recurrent Drafter (ReDrafter), an advanced speculative decoding approach that achieves state-of-the-art speedup for large language models (LLMs) inference. The performan…