Implicit Language Models are RNNs: Balancing Parallelization and Expressivity

Mark Schöne, Babak Rahmani, Heiner Kremer +3

State-space models (SSMs) and transformers dominate the language modeling landscape. However, they are constrained to a lower computational complexity than classical recurrent neur…