M${}^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling

Mayank Mishra, Shawn Tan, Ion Stoica +3

Transformers are highly parallel but are limited to computations in the TC${}^0$ complexity class, excluding tasks such as entity tracking and code execution that provably require gr…