Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting

Zida Liang, Jiayi Zhu, Weiqiang Sun

Transformer-based architectures achieved high performance in natural language processing and computer vision, yet many studies have shown that they have not demonstrated a clear ad…