2 papers
cs.CL2026
LinearARD: Linear-Memory Attention Distillation for RoPE Restoration
Ning Yang, Hengyu Zhong, Wentao Wang +3
The extension of context windows in Large Language Models is typically facilitated by scaling positional encodings followed by lightweight Continual Pre-Training (CPT). While effec…
cs.LG2024
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda +10
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios. Our approach addresses the challenge o…