Towards Understanding Transformers in Learning Random Walks

Wei Shi, Yuan Cao

Transformers have proven highly effective across various applications, especially in handling sequential data such as natural languages and time series. However, transformer models…