Curse of High Dimensionality Issue in Transformer for Long-context Modeling

Shuhai Zhang, Zeng You, Yaofo Chen +5

Transformer-based large language models (LLMs) excel in natural language processing tasks by capturing long-range dependencies through self-attention mechanisms. However, long-cont…