Longhorn: State Space Models are Amortized Online Learners

Bo Liu, Rui Wang, Lemeng Wu +3

Modern large language models are built on sequence modeling via next-token prediction. While the Transformer remains the dominant architecture for sequence modeling, its quadratic…