Separating Stream Stability from Long-Term Recall in Language Models

Peipei Cao, Xin Zhang, Jie Tang +3

Methods for streaming language models are often discussed alongside long-context and memory systems, although they solve different problems. An attention sink can stabilize autoreg…