PairConnect: A Compute-Efficient MLP Alternative to Attention

Zhaozhuo Xu, Minghao Yan, Junyan Zhang +1

Transformer models have demonstrated superior performance in natural language processing. The dot product self-attention in Transformer allows us to model interactions between word…