On the Computational Power of Transformers and its Implications in Sequence Modeling

Satwik Bhattamishra, Arkil Patel, Navin Goyal

Transformers are being used extensively across several sequence modeling tasks. Significant research effort has been devoted to experimentally probe the inner workings of Transform…