Implicit N-grams Induced by Recurrence

Xiaobing Sun, Wei Lu

Although self-attention based models such as Transformers have achieved remarkable successes on natural language processing (NLP) tasks, recent studies reveal that they have limita…