Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula

Sam Blouir, Jimmy T. H. Smith, Antonios Anastasopoulos +1

Efficient state space models (SSMs), such as linear recurrent neural networks and linear attention variants, offer computational advantages over Transformers but struggle with task…