Sparsified State-Space Models are Efficient Highway Networks

Woomin Song, Jihoon Tack, Sangwoo Mo +2

State-space models (SSMs) offer a promising architecture for sequence modeling, providing an alternative to Transformers by replacing expensive self-attention with linear recurrenc…