2 papers
cs.CL2026
From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
Yiqing Zhou, Yu Lei, Shuzheng Si +7
Managing extensive context remains a critical bottleneck for Large Language Models (LLMs), particularly in applications like long-document question answering and autonomous agents…
cs.LG2024
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda +10
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios. Our approach addresses the challenge o…