Transformers are Multi-State RNNs

Matanel Oren, Michael Hassid, Nir Yarden +2

Transformers are considered conceptually different from the previous generation of state-of-the-art NLP models - recurrent neural networks (RNNs). In this work, we demonstrate that…