On the Power of Convolution Augmented Transformer

Mingchen Li, Xuechen Zhang, Yixiao Huang +1

The transformer architecture has catalyzed revolutionary advances in language modeling. However, recent architectural recipes, such as state-space models, have bridged the performa…