Convolution for Large Language Models

Yuchuan Tian, Yingte Shu, Wei He +7

Large language models (LLMs) largely rely on Transformers, where self-attention provides global token interaction but does not explicitly encode the locality of natural language. W…