Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers

Sukjun Hwang, Aakash Lahoti, Tri Dao +1

A wide array of sequence models are built on a framework modeled after Transformers, comprising alternating sequence mixer and channel mixer layers. This paper studies a unifying m…