Stream-CQSA: Exact Out-of-Memory Recovery for Attention

Yiming Bian, Joshua M. Akey

Long-context large language models are limited not only by attention cost but also by out-of-memory (OOM) failures. A selected attention call may not fit in available device memory…